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ABSTRACT 

In the late spring of 1972, t,l^e Chancellor's Office 
agreed to support a summer study to be undertaken by a committee of 
the California English Council, to investigate equivalency testing in 
the area of English and to recommend an appropriate program for use 
by the California State University and Colleges. This report is the 
result of that study: it attempts to focus on the major issues in 
such a way as to p6int to their solution, and it recommends a method 
of equivalency testing in English which is responsive to the 
discipline and -practical to implement. The English examination 
administered in^nay of 1973 consisted of a 90-ininute essay test 
conducted and* graded by California State University and College* 
English faculty, and the 90-minute objective CLEP Subject 
Examination, "Analysis and Interpretation of literature, " which was 
also administered at that time. The most significant results of this 
study were that t,362 out of U,071 students taking the test were 
identified as deserving college credit for freshman English, and the 
essay- coniponerit has been shown to be a valuable part of freshman 
English equivalency testing. (The study and its results are described 
in both narrative and table format with newspaper clippings and news 
releases included.) (RB) 
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SUMMARY AND FACT SHEET 

This chapter, with the appended fact sheet, summarizes 
the results of the Freshman English Equivalency Examination 
that was administered to 4,071 students on ^May 12, 1973. 
* Analysis of the results shows that this program, designed 
and administered by the California State University and 
Colleges English Council.^ and funded by the Chancellor's 
Office division of New Program Development, was a major 
success; it provided clear and substantial benefits to the 
],362 students who earned six semester units of college 
credit, to' the English faculties ,^ and to the California 
State University and College system,. Furthermore, the 
project embodied a constructive resolution to what had - 
seemed to' be an irreversible and bitter conflict. 

For many years, English faculties have been largely 
opposed to the practice of large-scale equivalency testing 
and to the objective testing instruments generally used. 
This opposition was not without a basis io fact: most 
objective tests of writing ability, in particular, do 
not measure what is usually taught in^ freshman English. 

English teachers, then, have generally opposed 
external testing programs. These faculty have not had the 
funds, the time, or the specialized statistical knowled'ge 
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that are essential to develop large scale programs of their 
own. Thus,, it is not surprising that administrators, 
generally eager to proceed with equivalency testing, have 
been perplexed at the positions English departments have 
taken in this area. A particularly vivid example of this 
faculty-administration misunderstanding occurred in the fall 
of 1971, when large-scale equivalency testing was in fact 
instituted ^on two California State University and College 
campuses, arousing substantial ^opposition from English 
departments. \ 

The California State University and Colleges English 
Council took a statesmanlike position in the fall of 1971, 
rejecting what it called "an improper objective test," but 
endorsing "the principle of properly constructed and 
properly administered challenge examinations." Much 

2 

constructive discussion and committee work followed. By 
the summer of 1972, the Chancellor's Office and the 



For a convenient summary of the results of this 
testing program, see Urban Whitaker, "Credit by Examination 
, at San Francisco State," College Board Review , 83 (Spring, 
1972), 12-16. The conflict at its most heated appears in 
an exchange between Whitaker and Verfion T. Hornback, then 
president of the California State University and Colleges 
English Council, in The Future of General Education in the 
California State University and Colleges , Office of the 
Chancellor (1972), 59-76. 

2 

A committee 'on equivalency_testing established in 
the Chancellor's Office was under the chairmanship of 
Dr. Gerhard Friedrich; its sub-eommittee on English was 
under the chairmanship of Professor Charles Adams, of 
California State University, Chico. * Prof-essor James 
Clark, California State University, San Jose, was chairman 
of the English Council committee. The statewide Academic 



ERLC 



8 



Educatiopal Testing Service had combined resources to 

support an English Council report and proposal, calling 

3 

for a professionally acceptable examination. 

The success of the 'Spring 1973 test has implications 
that extend outside the field of English and beyond the 
borders of California; it has been shown that faculty, 
adminlstrators% and national testing agencies can work 
together in harmony for the benefit of students and the 
whole educational process, even when mistakes have been 
made that have produced an emotion-charged reaction. The 
^pro^ect has also shown that properly constituted faculty 
leadership, when given adequate .support, is an appropri=:te 
means of achieving academic innovation. 

The examination administered May 12, 1973 consisted 

of a 9'0-minute essay test constructed and graded by 

California State University and College English faculty, 

and the 90-minute objective CLEP Subject Examination, 

4 

Analysis and Interpretation of Literature. That objective 
test was the only such test to gain the endorsement of the 
California State University and Colleges English Council, 



Senate, through its Educational Policies Committee, chaired 
by Jerome Fox, California State University, San Francisco, 
also took an active interest in the matter. 

^ Edward M. White, Equivalency Testing in College 
Freshmah English ; A Report and a Proposal , 1972 . "Available 
through ERIC. (See Appendix, page ) 

^ The College-Level Examination Program (CLEP) is 
sponsored by the College Entrance Examination Board (CEEB) 
and administered by the Educational ^Testing Service (ETS) . 
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after careful scrutiny of objective tests available in 
1972.^ 

The results of the project are undergoing extensive 
analysis, particularly in the Chancellor's Office division 
of Institutional Research, which will be assisting the 
English Council in producing follow-up studies over the 
next several years. The report which follows contains a 
full account of the development of the project, including 
all information available by November 1, 1973. The most 
significant results now* apparent can be summarized briefly: 

1. 1,362 students deserving college credit for freshman 
English were . identified by a rigorous, responsible, and 
fair test. These students will be able to enter English 
courses at an appropriate advanced level, with positive 
feelings tov;ards themselves and their English studies, and, 
if they^^wish, to accelerate their college careers. 

2. Student writing has been shown to be a valuable part 
of freshman English equivalency testing. Those objecting 
to essay testing have generally cited problems of reliable 
grading and high cost. This project has demonstrated that 



A new CLEP S.ubject Examination in Freshman English 
has since been released , and it has received some f avorabJ^e 
comment. The new examination should not be confused with 
the CLEP General Examination in English Composition, which 
^as been declared inappropriate by conferences of English, 
teachers in California, Illinois, Florida, and elsewhere. 
A CLEP Sub3ect Examination in English Composition also 
exists, but it has be^n described as inappropriate by the 
CSUC English Council. It should not be surprising if 
confusion results from three CLEP tests in the same area. 
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both those o'bjections can be overcome. The carefully 
controlled essay reading conducted June 16-20, 1973, by 
seven'ty-f ive English professdrs drawn from all nineteen ^ 

California State University and College campuses, produced 

< 

highly accurate test scores at relatively modest cost . 

The high reliability of the essay scoring was a 
result of careful planning. The essay questions were 
composed by a representative and experienced group of 
English professors,, who carefully worked out agreen\ents 
on the nature of appropriate questions and on the criteria 
to be used in judging the candidates' writing. The readers 
were selected after nomination by department chairmen, and 
only regular faculty with a demonstrated active concern 
for freshman English were nominated. The. grading session - 
schedule contained time for frequent discussions" of standard 
using sample papers; the readers were carefully trained to ^ 
use common criteria for grading; four independent readings 
were given to each examination, and, where significant 
differences in evaluation occurred, papers. were reread to 
reconcile the differences; finally, there were systematic 
reviews and cross-checks on* individual ratings to ensure 
consistency throughout the reading. 

The basic argument for essay testing, however, does 
riot defend on statistical descriptions of test reliability, 
though essay testing can indeed be reliable. Almost every-v 
one will agree that a college student should be able to 
write with directness, clarity, and precision; obviously, 

11 " . 



anything motivating practice and instruction in writing 

serves fundamental educational goals. The existence of a 

valid and reliable test demanding writing, as part of a 

freshman English equivalency examination, is likely to 

I 

have substantial and positive effects on teaching and 
learning in the schools as well as in the colleges. 
3. Start-up funding for a similar test in 1974 has been 
provided by the Chancellor's Office, which has expressed 
the intention of exploring with the California Department 
of Finance ways of lncorpora<ting equivalency testing 
programs into the faculty workload budget. A preliminary 
analysis shows that- the credit hours earned cost the State 
of California much less than the usual expense^ for 
instruction. 

However, no one should assume' that the low cost of 
these credits means that the credits were cheaply earned. 
It is necessary to ,set high test , standards to make sure 
that students who have in fact not'gained college-level 
abilities do not receive college credit'; such students 
should not be deprived of necessary' educational experiences 
by equivalency credit awarded too easily.' Furthermore, it 
not to be expecte<^ that a *very high percentage of students 
without a college course should perform as well as 
students passing a year'^s'work in college English. The 
1973 -test has shown that, despite a rigorous test, graded 
with high standards, one-thii"d of the test group passed. 
In fact, the test group, perhaps because of the $15,00 



test fee, . seemed unusually able and hTighiy motivated; 
* thQse\who ^passed out-performed (on,, the essay test), most 
of the college students who had successfully completed .'the 
college ,course and who'se papers were graded by- the same 
standards. • * - ' . 

, - As long as the academic integrity of the testing 
process ^i§ safeguarded by responsible faculty direction, 
reasonable people can only, applaud the savirfgs ^that 
< equivalency ' testing offers, to colleges and to the able - 
and ambitious students who succeed. ' ' ' . 

4. The California State University and College system 

N^has attracted nationwide attention for its leadership in 
English Equivalency Testing. The policies and reports of 
the Engii>sh Council have been v/idely reprinted, and the 

-'^financial support of English Council proposals by the ' ' * 
Chancellor's Office has set new precedents. The director 
of the project has been asked to make major presentations 
at major conferences in .New York, . Lousiana , Illinois, 
Wa'shingto^, and Otah; conferences in Texas *and Florida ' ^ * 

- i ' ^ ' . 

have accepted and endorsed parts or the whole of the 
^* English Council report; newspapers and educational journals 
,*'«'rtave -Reported favorably on the project; and* ETS and CEEB 
^ h^vs shown an increasing disposition to follow the California 

model in the future. ' , ^ 



\ The norm sample of college essays' turned out'to be • 

unrepresentative of the California State University and 
Colleges system and could not be used for -setting cutting. 



•'5. Some results will be ha^rd or impossible to measure, v 
but are nonetheless* significant^ The existence of the test 
is likely to improve the preparation in English that 
secondary schools provide to their students; the test 
should 'help strengthen curriculum and encourage attention 
fo writing* skills. The existence of the essay grading 
session, bringing English professors from all campuses 
, together for evaluation of student, writing under special- 
>ized direction, is bound to imp2;^ve aspects of college 
Engl-ish teaching. Perhaps most important of all, th^ way 
in which equivalency testing shifts responsibility for 
leajfning onto the shoulders of students has large and 
positive implications for education as a whole, as any 
teacher looking at an essentially unmotivated class will 
agree. 

Finally, the director of this project wishes to thank 
the following ,/ whos^ assistance , cooperation , encouragement, 
and support' helped make the project a success :^ 

Members of the California St^te University and Coll^ge^ 
English Council, i\e., the department chairmen and freshman 
English coordinators, who gavfe substantial amounts of time 
aod, energy to the proj*ect; and particularly Drr -Richard Lid, 
California State University, Northridge, who served as co~ 
^director of the project, helping with all phases of the 



scores. * But the sample did show that students passing the 
test were performing better than many passing college students 

•'14 



9. 

work and handling with aplomb the 'tangled problejns^^of 
budget, facilities planning ,^ and correspojidence • 

Members* of the Calitorni^ StatjS'^ljn^iversity and ^ 
Colleges English Council committee that worked with the 
project directors to draw up the questions and grading 
criteria for the essay test: Professor Rex Burbank, 
California State University, San Jose; Professor Michael 
Caftwright, California State College, Bakersfield; 
Professor Will Crockett, California State University, San 
Jose; State University Dean Gerhard Friedrich; Professor 
Eileen Lothamar,. California State University, Long Beach; 
P'rofessor Bill Leary, California State University, Los 
-Angeles 

Members of the* Chancellor * s staff, particularly Dr» 
David Provost, State University Dean, New Program Development 
Dr» Jack Smart, Deputy Dean and Dr. David Leveille^ 
Associate Dean in the same office; Dr. Gerhard^ Friedrich , 
St'ate University Dean, Academic andi Resource Planning; 
D,r. Robert Bess, Director, Academic Projects; Dr. Leon 
Thomas, Associate Dean, Institutional JResearch; and Mr. 
Charles Davis, Public Affairs Associate. 

Test .Officers on the nineteen California State 
University and College campuses, who undertook unusual 
responsibilities for this program, particularly Dr. William 
'Abbott and Dr» Righard Cantey at California State 
University, Long Beach, who coordinated communications with 
th^ other test officers and assisted' the program in many ways. 
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The staff of the Educational Testing Service, 
particularly Dr. William Cowell, from the Princeton 
office, v/hose statistical assistance was invaluable; 
Mr. Alan Seder, from the Berkeley office, whose experience 
and- tact were equally invaluable; Dr. Richard Harsh, from* 
the Los Angeles office; Dr. Albert Serling, Program 
Director for the College-Level Examination Program in 
Princeton, and his assistant Ms. feetsy Barlow. 

i 
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1973 ENGLISH EQUIVALENCY EXAMINATION FACT SHEET 



Test Date: 
Test^ Used: 



Number of Candidates: 

Number Passed: 

Percentage Passed: . 

Number of Credit^ Hours Earned r 

Scoring Data: 

Minimum .Passing Scores: 
Objective Part: 
Essay Part: 

Combined Score : 



May 12, 1973 

CLEP Subject Examination, Analysis 
and Interpretation of Literature 
(90~mihutes) in combination with 

Two 45~minute essay questions 

prepared by CSUC Englxsh faculty 
(90~mj,nutes) 

4,071 

1,362 

33. .5% 

8,172 



Mean, Median, Mode, 
Standard Deviation : 

Objective Part: 

Mean : 

Mode : 

Median : 



45*(QLEP scale) (achieved by 60.3%) 

13. (out of a possible 24 points; 
equivalent to 48 on the^^CLEP 
scale. ) (achieved by 53.5%) 
* 

100 (49' objective + 51 essay, each 
on the CLEP scale.) (42.6% 
achieved 49 or better on the 
objective test, and 40.3% 
achieved 51 or better on the 
essay test.) (153 students 
achieved a combined score of 
100 or more but did not pass" 
because .they did not achieve a,- 
minimum '.pass on one of the parts.) 



47.448 

43.0 

47.149 
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Standard Deviation: 
Essay Part: 
Mean: 
Mode : 
Median : 

Standard Deviation: 
Combined Score: 
Mean : 
Mode : 

Median: | 
Standard Deviation: 
Essay Reading: 

Number of Readers: 

Number of Colleges 
Represented : 

.Grading Scale: 

Number of Readings per 
Test: r 



Total Essay Readings: 
Weighting and Scaling: 



Statistical Data,: 

Correlation' between CSUC 
Essay and CLEP Objective 
Tests : 



8.854 

47.916 
48.0 
47.662 
8.402 

• 95.363 ' 
91.0 
94 .485"' 
14.869 

Kellogg-West Conference Center, 
June 16-20, 1973* 

N 75 ' 

All 19 CSUC campuses 
6-pcint 



4,(2 independent readings for 
^ach question; additional 
readings to reconcile 
discrepant grades) * 

Approximately iO,000 

V 

The essay score was converted by the 
equipercentile method to the CLEP 
scale (20-8'0) , and the scores on 
both parts of the examination were 
added. Each part received equal - 
weight. 



.4777 



ERLC 



1^8 



13. 



Reliability : 

CLEP Test: 

Essay Test: 

Composite Score : 
Student Profile: 
Test Population: 

Female : 

Male : 
Pass Rate: 

Female : 

Male: 

Budget Data: 

Allotted h'y State of 
California^ (from New 
Program Development 
Funds) : 

Test Feesat $15.00 
per Student: 

Estimated Overhead and 
Miscellanecus'^Costs 
Contributed by' 
^ Institutions: 

CSU Northridge: 

CSC San Bernardino: 

CLEP: 

Cost to State of 
California per Credit 
Hour Earned.: 



.88 



,7183 - .1802 



.88 ? .05 



2,354, 97.1% age 19 or under 
1,702, 97.1% age 19 or under 

4 0.1% 
24.3% 



$64,003.00 



$61,065. 00 



$ 2,000.00 
$ 7,715.00 
$28,000.00 



9.00 (approximately) 
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III 

ADMINISTRATION OF THE PILOT PROJECT 
A. Personnel 

1. Project Directors , The October 26, 1972 California 
Stats University and Colleges English Council meeting 
concluded with the appointment of Professor Edward M, 
White, Chairman, Department of English, California 
State College, San Bernardino, as Director of the Test 
Program "for the English' Council • The council also, 
appointed .Professor Richard Lid, then Chairman, Department 
of English, California State University, Northridge, as 
assistant to the Director of the project. As the project 
developed. Professors White and Lid became co^directors , 
and? Professor Lid assumed responsibilities in the area of 
budget ,v' facilities planning, and correspondence. Since 
it proved more flexible to have the program budget 
established at the Northridge campus. Professor Lid also 
assumed supervision over the project secretarial staff and 
the^ disbursement of funds, ^ Professor White remained 
responsible for overall project direction, development of 
the test and test grading procedures, and relations with 
the English Council, the Chancellor's Office, and the 
Educational Testing Service. It proved to be a very 
useful division of labor for routine matters; however, on 
all important decisions, the project directors agreed and 
worked together. 

•20 . ■> . 



2. Coordinator of Test Administration > A series of 

additional appointments were made as the project progressed 

and as the need for them became apparent. In January, 

it became clear that the administration of the test itself 

» 

would require a substantial amount of time, training, and 
personal contact. Dr. William Abbott, Test Officer, Cal- 
ifornia State University, Long Beach, agreed -to act as co- 
ordinator of test administration, so that the nineteen sep- 
arate test offices which would be - administering the test 
on May 12 could have one central location for information 
and direction. Dr. Abbott and his very able assistant. Dr. 
Richard ,Cantey, relieved the project directors' of an 
immense amount of detail work and performed a valuable 
function in the cour-se of the test administration. Cer- 
tainly, any administration of a test on multiple campuses 
requires a similar diligent test officer /to coordinate the 
activities of the various test administrators. 

Among the functions that the test office performed 
are the following: 1) assistance in the preparation of 
a memo to all test officers (April 1973) providing basic 
information on the test program and requesting their 
participation, most particularly, the immediate selection 
of test locations on each campus; follow-up memos were 
sent on May 8, 1973 and June 10, 1973; 2) assistance 
in the prepar^ttion of the -t^est registration 
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form, and communication with the various test officers on 
proper ways to handle that form; 3) coordination and direction 
of all test materials in cooperation with the Educational 
Testing Service office in Berkeley; 4) preparation of a 
test manual 'for all test officers and proctors, in co- 
operation with the Educational Testing Service, Princeton; 
5) accumulation of information about the administration of 
the test for use by the project directors shortly after 
the administration; the California State University, Long 
Beach Test Office made approximately 200 telephone calls 
in connection with the project; 6) assistance in determing 
the statistical procedures to be used in combining scores 
and finding cutting scores. 

3. Statistician . As the test date approached, it 
became necessary to make several additional administrative 
appointments* Happily for the project. Dr. William Cowell, 
formerly Statistical Analyst for Advanced Placement, was 
made available to the project (at no cost) by the 
Educational Testing Service. Dr. Cowell was doubly useful 
to the project, which he served as Chief Statistician. 
His extensive experience with statistics in testing enabled 
the project directors to make wise choices on some diffi- 
cult statistical issues, most particularly the problem 
of combining essay and objective test scores and the 
problem of establishing fair cutting scores. In addition, 

the fact that Dr. Cowell was not only a participant but 

"> 

no 
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also an outside - observer helped to give the entire pro- 
ject additional credibility. It should be pointe<^ out 
that Dr. Cowell always functioned as an advisor to the 
project directors and to the State University Test Officers 
with whom the responsibilities for making decisions always 
rested. 

4. Essay Question Leaders . As the essay test began to 
be developed, it became clear that question leaders for 
each of the two essay questions would be essential, if 
the questions were to be created and graded according to 
the best available procedures. After considerable 
consultation, both within and outside the State of 
California, the following two appointments were made: 
1) Dr. Rex Burbank, Professor of English, California 
State University, San Jose; 2) Dr. Gerhard Friedrich-, 
State University Dean, Academic and Resource Planning. 
'Each of these professors of English has had over a 
decade of experience^ participating in controlled essay 
readings run by the Advanced Placement Program. The 
professional competence and experience of the two > 
question leaders were critically important to the sirccess 
of the test program. Their responsibilities were: 
1) participate in the development of J^e two essay 
questions ; 2 ) evaluate the results of the ^ pre- test of 
these essa^' questions and* report to the project 'directors 
with suggestions for revision; 3) assist in the selection 
of , table leaders and essay readers; 4) develop statemen"6s 
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for graders describing the criteria for grading; 5) select 
20 to 30 sample student e;ssays for d>Lscussion by the 
graders during the course of the reading and direct these 
discussions; 6) maintain supervisory control over the five 
table leaders and 30 to 35 essay readers in each question 
room during the reading of the' essays; 7) con'sult with the 
project directors during the reading on the schedule and 
overall administratio.n of the reading; 8) ^participate in 
the decisions on cutting^ scores; 9) prepare- a final 
report on^ their responsibilities during the reading, 
particularly with regard to the kinds* of distinctions that 
were made among the essays written by the students. 
5. Aides . Finally/ as the date'of the essay reading in 
June approached, it became necessary to appoint a staff to 
insure that, the test papers and other supplies were moved 
according to plan from reader to reader. Accordingly,^ a 
staff of 15 to 20 student assistants was selected for our 
use by Dr. James Ware/ Chairman, Department of. English, 
California State Polytechnic University, Pomona, and twO: 
chief aides were selected to supervise the student aides 
in their duties in the two question rooms and elsewhere 
during the reading. The chief aides were Ms. Linda Snyder, 
English Department secretary, California State College, 
San Bernardino, and Ms. Mary Reynolds,' an English depart- 
ment secretary, California State University, Northridgea 
Ms. Reynolds also doubled ^s project secretary before, 
during and after the reading, a second and fulltime 
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job in itself, (It would have been preferable tp have 
the project secretary remain in charge of secretarial 
duties during the reading, and have a- separate person 
acting as chief aide in charge of the student aides 
during the reading •) 

At the inception of the project, the project directors 
wire insufficiently aware of the extraordinary complexity 
of a large testing program, particularly one that included 
the creatic^, administration and grading of an essay test. 
Fortunately, however, as the project developed and new 
areas of responsibility emerged, it was possible to find 
extraordinarly well qualified people who were willing to 
assume responsibility for these areas and accomplish the 
necessary tasks in these areas with efficiency and 
responsibility. Without their assistance the project 
could not have .succeeded. Certainly, anyone undertaking 
to direct such a program in the future should be well 
aware of the spe.ciali2ed help he will need. 
B. Meetings and Decisions 

Beginning in October 1972, a series of meetings and 
conferences, sometimes as often as twice weekly, began • 
among the project directors sand a group of consultants. 
Out of these meetings a se'ries of decisions emerged 
which refined and developed the ideas contained in the 
English Council report of October 1972 | (See Appendix). 
The personnel and location of these meetings changed 
according to the agenda, but the following consultants 
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made themselves available for meetiags at^which their 
special expertise would be useful: 

From the Chancellor's Office: Dr. Da vic\ Provost, St^t 
University Dean, New Program Development and Evaluation; Dr. 
John Smart, Deputy Dean, New Program Development and Eval- 
uation; Dr. David Leveille, Associate -Dean, New Program 
Development and Evaluation; Dr. Robert Bess, Director, 
Academic Projects; Dr. Gerhard Friedrich, State University 
Dean, Academic and Resource Planning; Mr'. Charles Davis, 
Public Affairs Associate, Public Affairs. 

From the Educational Testing Service: Dr. Albert 
Serling, Program Director, College-Level Examination Pro- 
gram; Dr. Alan Seder, Program Coordinator, Berkeley; Dro 
Richard Harsh, Director of the Los Angeles office; Dr. 

i 

William Cowell. 

> - „ _ 

The project directors w.ere always^ plresent," repr:e^nir=^ 

ing the English depart^nehts ;. as the essay test b\fgan to 

be 'dev^op^d Drs. Frredrich and Burbank, along with other 

experienced English department representatives from the 

various campuses., attended several meetings. In addition, 

the project director was invited to spend several days at 

the Educational Testing Servrce ^office in Princeton, New 

Jersey tc consult with the various offices and personnel 

who have been planning and administering controlled essay 

readings for the Advanced Placement program for many years'. 

, While some of the decisions that emerged from these . 

advisory meetings may have^ b^en appropriate only for. this 
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particular testing program, it is likely that the issues^ 
and the way they were resolved will be usefuJ, as a r^f- 
erence for similar programs in various^ other states and 
under various other conditions • Thus, a summary of pro- 
blems and decisions follows: 

1. ' Single dosed Administration . Since we were to give' 
an es-say tegt and to publicize the .test very widely, it^ 
was necessary for security ^of the test to administer the 
te'st only once* Thu^-, we could not use the "open" College- 
Level Examination Program centers in tne state for our * 
purposes, since they offer tests monthly;" we.' needed to 
establish ohe clos,ed administr a'^ion of the test* We thus 
decided to offer the test on each of the nineteen campuses 
of the California State University and College^ system on May 
This decision worked out very well^ We we.re ab^le to main- 
*tain complete bqntrol over all processes of the test admin- 
istration and insure that test materials , including over 
4/000 essay examinations, arrived at th^ proper place at 
the proper time to be prepared ^or the^grading session. 
.There were a few complaints from students who could not 
take the test the one day it was offered, but the many advan 
tages of a single closed administration of this sort of 
test' became apparent Very quickly. Ou>r conclusion was that 
under similar circumstances a single closed administration 
is" by far the best way to proceedo 

2. Location. Sinc§ the California St^te University and 
College system was sponsoring this particular test, we 



decided that the' test centers should be at -each of the 
nineteen campuses in the system rather than at various 
CLEP centers around the state. This decision .worked ver-y 
yell also. The test was clearly and publicly a Qalifornia 
State University and Colleges test, not an ETS operation, 
and the test program redounded to the credit of the Cal7 
ifornia State University, and Colleges system. In addition, 
we' needed to deal only with California^ State University 
and'' Colleges test of ficers , instead of the variety, of 
personnel who aTe directing CLEP centers in various loca- 
tions around the state. A few difficulties did arise: at 
the Pomona c^pus, there, was a conflict with a large 
campu3/ activity, and only with some difficulty were' 'facil- 
ities found for the test. On a few other campuses oth^ere was 
an uncpmfortable moment or two, as -enrollments for the test 

■began to accumulate an<3 facilities were limited. We' did - 

\ 

make plans to re-direct students, in case some campus 
facilities were over-taxed, but in no case did it become 
necessary. The test officers or/ e'kch campus were coopera- 
tive and wholly competent to arr^rfge for the machinery of 
test taking at each of -their .locations. 

3. Day of Test . We had originally thought that a- Thurs- 
day might be mojfe appropriate than a Saturday since some 
stuc^ents may have religious objections to being tested on • 
Saturday, or be otherwise unavailable on a^ weekend. These 

reasons were counterbalanced by a possible -shortage of 
* 

seats available for testing on a Thursday , when classes are 



in session, and by the need for high school students to 
obtain special permission to take a test^ at college on a 
school day. We decided therefore to offer the test on a 
Saturday, and that decision appears to have been wise. We 
beard no ^complaints whatsoever from students on religious 
grounds, and the test officers found themselves much more 
able to deal with problems of administration on a Saturday, 

4. Date of. Test . After careful consultation with repre- 
sentatives from the Educational * Testing Service, the test 
date selected was Saturday, May IZ, 1973, It was necessary 
to check, carefully to avoid ^conflicts wLth other examin- 
ations the same students might wish to take. We were par- 
ticularlv catreful to avoid conflict with Advanced Place- . 
ment, since we suspected there would be students desiring 
to take bo.th of these tests. 

5. Time of ^^Day . After considerable discussion, it was 
decided to offer the examination from *10 : 00-12 : 00 noon 
and from 1:30-3:30 p.m. 'There was" some opinion that the 
test should be -given in one block of time, say from 9:00- 
i-2:30 p.m.. Such an arrangement may well have been more 
convenient for some students and proctors, but we decided 
on the later starting time in order to. help students who„ 
would have' to travel long distance's; and we decided to 
break far Ivii^c^,^ between the objective and essay tests, so 
that student^ would be refreshed and thus ready to write 
better ,e«s"ays. (The t'est" 'of f icers' were instructed to pay . 
proctors on the basis of a single test session, even though 
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it was divided by a launch break, which resolved one diffi- 
cult problem in'^payment of proctors •) On the whole this de- 
cision proved to be successful, but there were problems. 
Sdme. students showed up for one session or the other, in- 
stead of for both, though that may in part have been 
attribu1:able to an unfortunate typographical error which 
appeared at one point in the registration form. A future 
administration should make clear that* attendance at both 
parts of the examination is required in order for the exam- 
ination to be considered. At least one campus took advantage 
of the lunch Soreak to show some hospitality to the students 
on campus, offering them free soft drinks y to go witlr their 
bag lunches. An enterprising grampus , seejcing to recruit 
unusually : able students', could ^.ell offer the students 
additional hospitality.- It is, of course, impossible to 
say how much the bteak between sections influenced the 
overall quality of the .essay test, which was very high. 

6. Proctors . Each of the nineteen test offices desig- 
nated a person to act as chief proctor on that campus and 
the chief proctor appointed sufficient assistant proctors 
to administer the examination in accordance v;ith normal 
ETS procedures. The coordination and direction of these 
personnel were carried out efficiently through the California 
State University Lortg Beach^ test office. 

7. Payment of Proctors . Honoraria were paid to. the 

California State University and Colleges institutions by 

CLEP according to i ts regular honoraria schedule . These 
"I 

30 ^ ■ 
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fees were part of the $15 test. fee paid by -each student and 
proved adequate to support the expenses of administering 
the entire examination. ETS would have been willing to 
send a single check for proctori;ig to the directors of the 
program, who would then pay the various test offices. But 

this idea was rejected as unnecessarily complicated. Each 

I* 

test office submitted to ETS its roster of students taking 
the test, and ETS sent checks to the individual test offices 
on each campus. 'Though there was some small delay In receiv- 
ing payment, the test offices found this arrangement satis- 
factory. 

^8. Movement of Test Materials . This very complicated 
matter was handled with great efficiency through the ETS 
office in Berkeley, under the direction of Alan Seder of 
that 'office. All test materials were received in the Ber- 
keley office in ample time so that they could be sent from 
Berkeley to ^ the test centers 10 days! before the administra- 
tion of the test. By ,that time each iiest office knew the 
number of registrants hnd had reportedX that information to 
the test office at Calif ornia ^ State University, Long Beach; 
also by that time the essay questions had been printed and 
sent to Berkeley from Northridge, and the special test man- 
ual had been printed and delivered. Mr. Seder included an 
overage of 15 percent to accomodate fate registrants, and 
all materials were received by the > test;- centers in ample 
time. At the conclusion of the test, the essay examinations 
were mailed to Northridge by each test center; the objec- 



tive tests, answer sheets, and all other materials were- r.e- 
turned to Priacetpn.. There were no mix-ups in the delivery^ of 
material, and the o^ly difficulty was the i^te mailing to 
Princeton from^ several of the test offic;^s, which failed to 
get the^ materials in the mail, in some cases, until Wednes- 
day or Thursday of the week after the test. Nonetheless, all 
answer sheets were handled by the Princeton ETS computer 
according to schedule, and the objective score report was in 
the hands' of the administrators of ^he program in time for 
the essay reading. ' » . ' , 

9. Closing Date and Late Registration . The announced 
closing date, by which time all registration forms were to 

be in the hands of test officers, was April 27, two full weeks 
before the test date.* The test officers did, however, ac^cept 
registrations up to within a few days of the test at their 
discretion. The established .closing date worked very well 
to pro.vide us, with the necessary information about the amount 
of test materials to have available at each test center. 
There were few enough late registrants so that they caused 
no particular problem, and we received no complaints from 
test officers about this procedure. 

10. Test Manual . The original plan to adapt a CLEP.. test 
manual, with a small supplement of our own, became imprac- 
tical as we discovered the many differences between our test 
and usual CLEP procedures. It was thus decided to prepare 

a test manual specifically for our test administration. With 
the advice and cooperation of Alan Seder, in Berkeley, and 
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Ms. Betsy^ Barlow, In Princeton, who bore particular re- 
sponsibility for the preparation of the test manual, this ^ 
manual- was prepared, approved, and printed in time to.b^e 
delivered to the test centers with the test materials. Dr. 
Richard Cantey, of the California State University, Long 
Beach, test office, took particular responsibility for over- 
seeing production of this manual. The preparation of a test 
manual should be Considered as one of the special respon- 
sibilities of the coordinator of test administration; and 
it would be well to prepare this manual well in advance, so 
tha\ the condition of urgency which prevailed during tthe 
last few days of production in 1973 need not occur. 

11. Test Form . Dr. Albert Serling, Program Director for 
vCLEP, gave instructions that the particular form of the CLEF' 
Subject Examination, Analysis and Interpretation of Literature, 
which would be used in the May 12 test program, not be offeree^, 
in the state of California through the regular CLEP test 
centers. This was ,a wise precautionary move to insure the 
security of the test program. 

12. Registration Form . We could not use the standard CLEP 

i 

Registf ation-Information form, since it asked for information 
of no use to us, and failed^ to provide the necessary in- 
formation' for students taking the examination. The produc- 

tion of a new registration form absorbed a considerable por- 

< 

tion of the energies of the test administrators, and became 
the major vehicle of information about the test throughout ^ 
the state. Since the form needed to be printed in great 

33 
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haste in order for the publicity program to get under way, 
it was not possible to use the Office of the State Printer 
to prepare the form, and one unfortunate typographical error 
did appear in the final printed copy. Nonetheless-, an 
excellent printing job was done by the Franklin Press in San 
Bernardino, and almost '60,*000 copies of the form were dis- 
tributed throughout the state, A repeat of the testing pro--^ 
gram needs to insure that the registration form be prepared 
well in advance, thoroughly and carefully, 
C. The Objective Test 

The 1972 report on equivalency testing pr'fesented the 
arguments for the use of the particular objective test that 
was used in the test administration May 12, 1973, and there 
is no need to review those arguments here^ The California 
State University and Colleges English Council agreed in 
October 1972 that the CLEP Subject Examination, Analysis and 
Interpretation of Literature, was the most appropriate ob- 
jective test available at that time for freshman Eng- 
lish equivalency. This test is basically a good reading 
test: it contains approximately 100 multiple choice ques- 
tions, all of them based on passages supplied in the tGst, 
The passages are selected so that no previous experience with 
them or knowledge of their background is required to answer 
the questions, and the passages are taken from 19th and 20th 
century American literature and from each of the major periods 
of English literature from the Renaissance to the present^ 
The results of the test show that this objective test was 



appropriate for the test giroup, and served effectively to 

make distinctions between those with colle^ge-level skills 

in reading literature and those without such skills. At the 

same time the correlation coefficient between the objective 

^nd the essay test (.477) shows that a substantial portion 

of what was. tested in the objective test was not tested by 

the essay portion. Further evidence of the validity of the 

objective test appears, when we notice that each essay qu^es- 

tion correlated more highly with the objective test than with 

the other portion "of the essay t,est. 
-» 

There can be no question about the use of objective 
testing in large scale English equivalency tests; an ob- 
jective test is essential. This particular objective test 

worked very well and may be worth using again for that very 

''a • 

reason. The major objection to the use of this tfest relates 
to the nature of the freshman English course itself. In 
many cases, essay writing in freshman English uses litera- 
ture as its subject. In the May, 1973, test, since half 
of the total examination, the 90-minute objective test,, was 
a test on the reading of literature, it was felt necessary 
that essays be written on non-literary topics. If a good 
objective writing test that would not be based upon an 
ability* to read literature were. to be used, it would then 
be possible to give ani essay test on literature; such an 
essay test would strike many English professors as still 
more valid than the 6ne given in 1973. 

A new objective CLEP Subject Examination irv freshman 



English is now available.^ If this new te3t is used' for the 
19.74 examination, it /would make it possible for the essay 
test in 1974, to-be about literature. The statistics avail- 
able from ETS on this new test indicate that it accurately 
reflects achievement in freshman English courses in a wide 
; variety of institutions, including several in the California 
State University and Colleg.es system. But the administrators 
of the 1974 test will need to decide whether a new objective 
test, which allows them to use an essay test on literature, 
is worth using, when the available statistics from 1973 show 
that the CLEP Subject Examination, Analysis and Interpretation 
of Literature, accurately measures important aspects of 
' freshman English skills. 

Meanwhile the Advanced Placement Program has shown con-- 
siderable inteifest in the California' State University and 
Colleges Eiiglish Equivalency Test, and the College Entrance 
Examination Board is considering possible revisions in the 
"structure of its several college equivalency programs. 

Certainly, the administrators of the 1974 .examination 
need to consider all available instruments before a decision 
is made about which objective tes4: to use. 
D. The Essay Test 

1. Format . It was decfided that the essay examination woul'd 
be 90-minutes long and t)e weighted equally with the 90 minutes 
o'f objective testing. There vas considerable discussion of 
weighting the results differently, but there were as many 
"arguments for weighting the essay more heavily than the 
object^ive test, as there were for weighting the objective , 
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test more heavily than the essay. It was finally the. con- 
sensus of all those involved that the tests should be of 
the same length and should receive equal weight. Various 
ways of distributing the 90 minutes of essay testing time 
were discussed, and at first it was proposed that three 
30-minute essays be used, each essay to receive one reading. 
After further consideration and discussion, it was decided 
to 'give the 'students . longer time to reply to two different 
questions, and, thus, to require tvo 45-minute questions. 
If each of the two questions received two independent read-- 
ings, it seemed apparent that the reliability of reading 
would be increased and that a more fair score would be ob- 
.tained. Thus,^ttie test was made up of two 45-minute ques- 
tions, each testing different kinds 'of skills. It was, also 
decided that the time limitation would be enforced Mo that 
graders would be evalua"ting comparable effort; after 45 
minutes, question one would be collected and question two 
distributed. 

2. Creation of the Essay Test . Early in the planning the, 
decisig^n was made to create our own essay" test, and not use 
the literary topics provided with the CLEP Subject Exam- 
ination. Analysis and fnterpretation of Literature. Since 
90 minutes of objective testing were on literature, .and since 
most freshman English courses in fact require writing on 
many subjects besides literature, it seemed f^jfe^ to provide 
noh-literary topics for the essay test. Once the decision was 
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made to create our own essay test, it was necessary to 
decide precisely .what kinds of" skills we were seeking to 
test, so that we could examine those skills. This task 
was acconjplished by a committer that met for a full day 
with the project directors and the question leaders. (See 
Chapter 1, page 9 for the members of that committee,) 
After prolonged discussion, the committee decided to 
test on the first question the student's ability to des- 
cribe an object from persoftil experience and his capacity 
to move from description to abstraction. The second ques- 
tion wO,uld ask for a response to two short passages, and 
demand the ability to respond incisively to others' ideas. 
The assumption was that essays emerging from personal ex- 
pe^rience cali for quite 'different kinds of writing ,gkills 
than essays comparing and contr^asting quotations; both kinds 
of questions are common in fxeshman English courses, (This 
assumption was born out by the results; the correlation 
of question 1 to question 2 was only ,3681o This relative- 
ly low correlation, despite the very high reliability co- 
efficient of readers for the same question, supports the 
sophisticatibn of the essay reading, which clearly did v 
not give a high priority to elementary matters su.ch as 
spelling, which may be presumed to be consi:ant on both 
essays, ) 

3. Grading System , After considerable, discussion of the 
various grading scales that have been* used in i^he past, 
it was decided that grading would proceed on a 6~point' 
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scale, ranging from 1 (lowest) to 6 (highest). The 6- 
poinf scale was chosen because it Liberated readers from 
conventional A, B, C, D, and F grading p'attern,^ and, hence, 
would make. them more> likely to abide by group standards, 
a'nd because the 6-point "scale contains no middle score. " 
The graders are^, forced to decide between a paper that is 
in the top half (4,5,6) and,, thus, a passing one, or in the 
bottom half (1,2,3) and, thus, .failing. The 6-pdint scale 
is really a pass/fail scale, and the basic decision is 
between a. middle pass (5) and a middle fail (2); since 
an equivalency -test renders a pass/fail decision, the 6~ 
point scale was more appropriate than any sc^le that con- 
tained a middle score. At the same time, simpler versions 
of the pass/fail (a 2- or 4-point scale) seemed to allow 
insufficient scope for student variation. The results of 
the use of the 6~poant scale were very satisfactory, and 
we recommend that such a scale be used for such tests in 
the futui;e. When the four grades were added, the lowest 
possible score < for a paper th^t respo^ed to -both ques- 
tions 'was a ,four, and th^e highest was a 24. The results 
of this jscale are set out in considerable detail below. 
All t^'^t need be said here is that the 6-point scale work- 
ed very well for the purposes of '"this examination and 
allowed us to make the distinctions that were needed with 
confidence. 

4, Question 1 . The following question was* distributed 
to all students: 
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"Many obsej^vers of ,our society claim tha.t modern man^ im- ^ 
mersed in materialism, is "owned by his objects." Yet many 

V 

of us have o^bjects that we treasure not just for their ma- 
terial value but for a variety of other reasons As^gnm^nt : 
Describe one or mere objects which are^lmportant to yoU. 
E xpLain what values they represent and comment on those 
valuek-ii— ^ ' .... 

The following directions for iscoring were distributed to 4 
all readers engaged in the grading of question 1: 

"The student should be rewarded for what he does well* in 
response to the question. Here the student is asked to 
describe one or more objects important to him. He is fjurther 
asked to explain what value s they represent and to comment 
on tbc^>se values. He is told to think about the question and 
to plan his response. 

""Note that the question does not demand that the object or 
objects be unusual ones. 

"Essays which mi'sinterpret "objects" as "objectives^" and 
which deal mainly with generalized abstractions (life,, 
God), should be read sympathetically, but they should or-"" 
dinarily ,not receive above 2, since, they fail to under- 
stand and properly respond to the question. 

"Possible Scores : ' ^ " ' ' 
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6 A superior response will not just name one' or more 
object but describe them in some detail, and it will 
not just identify the values represented but* explain 
and* comment on them, thej,r nature and their source. 
A superior paper will be literate and orderly. 

5-4 These scores *.;ill be useful for a well-handled papei" - 
which -is deficient in one or two characteristi,cs of 
the superior response, i.e. in description* of the 
object or objects and in explanation of the values 
represented, but which 19 otherwise -competently written.* 
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3-2 These scores will* be useful for the following kinds ^ 

of papers: , , . • ^ 

— those in which only one part^of the two-part question 
is addressed; ^ ' - 

^ -^those in. which the representativeness of specific"* 
^ objects is ignored; 
^ — those which treat the subject in superficial or v 
stereotyped fashion; ' 
— those in which' the writing exhibits several weak- 
nesses, in wording and otherW respect/. ^ - ^ 

*1 This score is to be used for papers which a,re lack- 
ing in focus and substance, depart from the assTgn^ed 
tt pic,' and/or •exhibit serious writir^^g faults. 

* Non-response papers and papers which do not^fall into 
the fbregoing categories, extensively, argue wit'h th^ 
question, or are otherwise idiosyncratic, should 
.immediately be brought to the attention of the ta'ble, 
reader and the 'question leader." 

The following report was prepared by Dean Gerhard Fri^drich, 

question leader for the first question: 

* ■> 
SUMMARY REPORT ON QUESTION #1 

For the first, experimental administration of^a system- 
wide, equivalency test in Freshman English,- it was , decided 
to employ a^ ninety-minute composition portion consisting 
of two dis-tinct writin^g tasks. The first essay question , 
would be relatively open-eroded, permitting the candidate ^ 
to start wtiting without any "hang-ups''; the second essay 
questi'on would be more structured, requiring the candidate^ 
to "deal With a given subject. • ' 

A i^Ianning committed called by the co-directfors considered' 
a variety of test questions. Two were eventually selected 
for careful rewriting'/ pretexting, and further refinement. ' 
.A special effort was made' to assure 'that the phrasing of 
^^the questions would be absolutely clear, with the praptices 
of the Advanced Placement English examination as a guide. 
Pretest responses from students in comparable California 
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State University and Colleges classes were reviewed to 
determine the actual/suitability of the proposed essay 
questi6ns and particularly tl;^e wording of the instructions* 
The entire composition portion Vas .'then" administered to a 
larger sample of California State University and Colleges 
students"; ^hopefully for norming^ purposes • 

After the candidates' essay , booklets had been received 
, from the test center.s, the two question leaders read a 
lajrge number of responses from^all the test centers and 
selecteji sample essays -illustrating the range of perfor- 
mances to be identified, from excellent to incompetent* 
It was '(Jecided to employ a 6--point scale, ^with scores of 
6, 5 and 4- indicating degrees, of creditable performance, 
and scores of 3, 2 and 1 degrees of deficiencies in terms 
of college-level composition. Both .question leaders par- 
ticipated, immediately before the reading of the English 
Equivalency Test at Kellog.g-West , in the Advanced Place- 
ment English reading at Rider College, and spent consider- 
able time in drav/ing up "rubrics," i.e. guidelines or 
criteria for-scoring essay responses, in accordance with 
Advanced Placement models. Copies of the final version of 
Ques1:ion #1 and of -the instr.uqtions to readers of Question 
#1 responses are attached for reference. 

All readers of the candidates' compositions were drawn 
.from among' thq English faculties of the nineteen campuses 
in The California' State University and Colleges. Readers 
were .divided into groups of six or seven, each with an 
experienced table Reader. The table leaders for each 
question were brought together in a pre-reading session to 
harmonize grading standards ^on the basis of a representa- 
tive sample^tf papers previously selected . Subsequently, * 
together wfxfr' their question leader, they similarly in^ 
structed their respective readers. Consistency of standards 
was further ensured by having the table, leaders regularly 
double-check scores assigned by readers; readers were^also* 
encouraged to confer with their table leaders on any 
sco-rin§ problems. Qu'estion leaders in turn double-checked 
the scores assigned by table leaders, and from time' to 
time throughout .the 4r'eading polled the entire group on 
additional sample papers. In this way, a workable con- 
sensus was -ra.ther easily achieved and maintained. In the 
relatively f ew^ instances in which the scores assigned 'to a 
Question #1 *essay . were at least two\points apart, the essay / 
'received a thi^d reading, followed by discussion and appro- , 
priate^y^d jusj:ments. , * ^ ' ' 

« 

It should -be nqted that** the reading of Question #1 essays 
was both f air-mi/idedly-^esp^nsible and very efficient.. 
-Readers took the t:ask of assigning appropriate scores 
seriously and manag^ to read ^more*' essays per day than 
had been anticipated. The candidates! compositional abilities 



covered. an amazingly- wi(3e range, from brilliant to illiter- 
ate, and in most instances the readers of the same composi- 
tion assigned identical scores. Moreover, in many instances 
a candidate performed at the same level in response to 
both essay questions » However , there were.al^o striking 
exceptions, when no. correlation wa^ apparent between a 
candidate's responses to the two essa^^j questions, nor be-^, 
tween the essay portion and the performance on the objective 
test. ^ 

Attached are examples of candidates ' responses to Question 
#1, illustrating the variety of objects chosen as well as 
the scored levels of performance • 



Gerhard Friedrich 
June 30, 1973 



The following student responses to. Question #1 were sample 
papers used during the -reading to illustrate the grades on 
the 6-point scale: 

SCORE OF ONE 

"I am going to write on some objec't which are pretty im- 
portant to me. The first would be my religon because I think 
I should put my faith in something other than 'man', and^ 
worldly goods. I also believe that I as a person have that • 
right to look forward to something bigger and better in life, 
I an live life better day by\^ay. This is important because 
I'm a person .who doesn't like to have things cramed down 
fny throat then- expected to- digest it easily. I feel I have | 
the right to choose who and -jwhat I believe in, without some- 
one handing me 2 alternatives to chose from. I want to 
choose freely on my c?wn will and judgement. I feel that I 
am old enough to choose and have 'free choice*. Another - 
pbject or value which is important to me is my 'Freedom'. 
I like to do* v;hat I like, go where I want, see what I want, 
to a point where I, do not interfer with another persons 
'Freedoms.'' My whole li'fe is based on what I can do for me 
and other people.- Without this freedom I could not serve 
to the best of my ability my fellow man. I couldn'^t put 
forth 100% because I would be restricted to do only certain 
things. Therefore, without my religon^ and freedom to do jj 
what'I want I should have never have been born and with, out 
these two basics pf life I do not see how any man can live. 
So I have told you two* of my basic values, again they are 
'Freedom- to choose my own religon' and basic .Freedom to live 
an everyday life." 
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SCORE OF TWO , 

--"Heat, exhaust, fumes, burning rubber and smoke are all 
caused by a remarkable invention that, has spured our society 
into being one of the most materialisf ic in this modern age. 
Our society today depends on the car for transportation. We 
overlook the bad side of this invention for all the wonder- 
ful^ 'things the car has done for, us. 

No longer are people confined to one small region for 
their entire lives. Trips to the coast or to a distant city 
for a day are not unheard of now. It has actually broadened 
our horizons for we can meet new people, go new places. 
People we haven't seen in along time are in easy reach. 

Working days are shortened with the use of the, car. In- 
stead of walking many miles to work, it provides fast and 
easy transportation on highvyays. 

Status is related to owni^ng a car. Some people seem 
to feel that" the bigger a car is, the better it is. Socio- 
economic status is based on the number of high value mater- 
ialistic things we own. Having four or five cars in a family 
tends to rais^ a famiJ.y*s status. 

Cars come in all shapes and sizes. Big or small we can 
find one that fits the needs of everybody. Cqmpact, economy 
and luxary cars are priced to fit people with even low in-^ 
come budgets as well as high. 

There Xs a limit that people using cars must dr^w. Ex- 
cessive use of a car can damage our enviornment. Taking, a 
car into high mountain area can damage or even ruin flora 
and fauna. Pollution from cars cannot be stopped unless all 
cars axe banned. 

With all the good and bad sides to cars, which way 
can, we turn? Cars can be used for destructive purposes as 
well as useful, meaningful reasons." 



SCORE OF THREE 



"Bear-Bear" . 

'•^lue body with a white tummy and round black eyes,, 
soon to be loved. This poor little teddy bear went through 
so much just for ihe. 

When I came home after my birth, I had 2 sisters and 
a strange new friend waiting for me. My sister Monnie Leigh 
had a teddy bear placed in my crib. They tell me I actually 
giggled when I first saw my^ teddy bear. 

Well day| went on and after countless washings , due to 
being thrown-up on or thrown into the toilet, my little teddy 
bear had seen his last day with those round black eyes. Yes, 
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my little teddy bear needed some new eyes and blue buttons 
seemed perfect. After all, blue eyes match a blue body I 

Monnie Leigh decided , one day', that she vould*" teach 
me how to say Teddy Bear. It was a noble effort on her 
' part even though all I oould repeat was Bear I The Teddy 
bear still didn't -have a name. 

The next day when I was in the living room, I realized 
that my. teddy bear wasn't around. With the terrifying 
thought that he might be gone, my tiny voice piped up with; 
•Bear-Bear^ Bear-Bearl' • - > ' 

There was no mistaking what I wanted. Now that he had 
a name, Bear-Bear would never be more thanja helping hand 
^way. , ^ / 

I never really pondered on why i kept/ Bear-Bear all 
these years. It's simple - I Ibve him and /I'll keep hi^m 
many more years I imagine. Bear-Bear will/ always me^n love, 
security, and friendship. Perhaps .that ' s'ywhy I choose my 
friends Ccfrefully and valu^ their love so greatly. My 
friends have been wonderfully good to* me (and vise- versa) 
since »the very first time my giggle said 'Hi, let'.s be 
friends! ' " 



SCORE OF FOUR 



' "At this point in life, I don't have many objects 
which I value a great deal. The few things I do value have 
a -sentimental value rather than a monatery value. 

The first objects I value are trophies I won diving. 
They have a very deep sentimental value to me because it was 
my rteward for the hours and years I practiced, working for 
a go^l. When I look at my trophies on the shelf I think of 
all the joys . I felt at winning and also** the heart break of 
losing. I think of the self-control and self-discipline J 
gained at going- to practice each day while my friends were 
at the beach. Thi's, right now, is very important to me. 
These trophies have no xeal monetery val.ue or sentimental 
value to anyone but me because only I. earned th^se trophies 
* snd only I cherish them. 

Another object which I value is a ring I recieved from 
my gra'ndmother . This ring is also a sentimental obje'ct to* 
me. I recieved it' after she passed away and so it is my 
rememberance of her. This ring also has monetery value. It 
is a gold ring with a small diamond in it. '^Others would 
value it because of it's worth but my family and I are 
the only one3 who value it for a sentimental rpason. 

My next valuable object is my wallet. My value on my 
wallet is very sentimental . In it , it contains all my 
pictures of friends and experiences which I have gone 
through. If someone stole my wallet or 'I lost it, I would ^ 
rather lose my money than some of the pictures inside. 
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Maybe this is because I am not overflowing with money at 
the moment but right now my pictures come over my money. 
* ' My last valuable, is a watch I received at Christmas ^ 
from my boyfriend. This is very important to* me because it 
is from him. I have had watches before from my parents 
and^ they really didn't ha-ve very much sentimental value to 
them. But^my watch contains many memories and I would really 
be upset if-I misplaced it. Just like the ring, it is .val- 
uable to others because bf the. cost but to me it is the 
person who it signifies. 

These are my most valuable objects and it isn't be- 
cause of the amount of money they're worth but instead for 
the sentimental value of therft. I don*'t really own anything 
excessively expensive so I really don't value many things 
for they '-re, value in money." 



SCORE OF FIV^ 



"As I look^ back'on .my life, the* object that I place 
the most value b^t^^s the house that I grew up in.' For six- 
te^n years I walked through its doors and lived in its rooms.; . 
That house' became a pajrt of me. 

Now, almost nineteen years old, it stands in a middle- 
class suburb of Los Angeles. The surrounding streets are 
lined with well-rkept homes and neatly- trimmed yeards. 

Children that I don't know play baseball on the 
avenue, and cars that I don't recognize fill the driveways. 
My dear house is in an alien world both to me and to it. 

My family took pride in that home. We .bought it ^new, 
put in all the landscaping, and made it a beautiful place 
'to live. In all the years, we lived there, I never once 
took its loveliness for granted. I would sit and look at 
it and know what a wonderful home we had. ^ 

That long avenue was my world. Little playmates 
moved in and out of th*e other houses, but I was the stable 
one. I didn't believe we would ever leave our home. 

That building saw my first step, heard my first word, 
and watched me f^ll off of my first bicycle. It stood by 
when I was sick, and was. there for all the happy moments 
too. It became more like a person, part of the family. 

Leaving it all alone for new people to run about in 
was next to impossible. Is it ^s lonely as I am? 

Our new house is bigger and more modern than that 
one was. Still, this makes no difference. It will never 
be home. " 
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SCORE OF SIX 



"We h^ave in our living room a music box, which for 
three generations has given pleasure to the eyes and ears 
of my family. It stands about a foot highland measures 
about two feet in length and width. Except for a spray of 
flowers carved on its face, the outside is unadorned. In- 
side, pasted, to the lid, is a turn-of-the-century litho- 
graph of a pair of plump cherubs. There is a .set of tin 
records, perforated here and there, that goes with the 
music box. It is run by winding it up and releasing the 
spring. Its tunes are dated; 'My Gal Is A High-Born Lady* 
and 'I Guess. I'll Telegraph My Baby', haven't been" ampng 
the top ten for quite a while, but this d,oes nothing to 
lessen the enjoyment they give. 

My grandfather was the first to own the^ music box. 
He traded a horse'' for it and presented the music box to 
my grandmother as a gift. They had been married for only 
six months. At first it was a very big deal. A music box . 
in a Nebraskan farming town can cause quite a commotion, 
but as time went by and the popularity of 'victrolas' 
grew, thle music box passed into oblivion. ^ 

Ignored and dusty was the way my* father discovered' it 
in the cellar. He cl;eaned it up and got it running and 
showed it off patronizingly to his friends as a relic 
from his parent's youth. It was played at parties as a 
.novelty, but again it lost out against the incoming rage: 
the radio. So back into the cellar went 'the music box 
to await rediscovery one more time. 

This time it was my sister and i who resurrected it. 
We hauled it out into the light., dusted off its rosewood 
sides and listened to the songs first heard what seemed 
to us to be eons ago . 

My grandparents- grew old, and being practical people,^ 
decided to divide their possessions with their children 
^before their death to avoid a tragic scrabble afterwards. 
*To my father went the music box, and he carefully brought 
it to bur home and revived it one more time. 

Now, despite its years, it keeps on playing its old 
familiar songs. I love the old music box. It can never be 
asso'ciated with a pr^ice tag. My grandfather acquired it 
with an honest trade and it has been handed down through 
the years. The music box symbolizes my grandfather's iove 
for my grandmother, my father's years at home, my sist^er 
and I exploring in the dark cellar and countless fine 
memories, i love it for its beauty, the rich, soft red 
of the rosewood, the way it gleams in the sun. I love the 
whirr of the motor and vigorously cranking the handle. I 
love the corny .song titles and running my fingers over 
the rough surface of the records. And although its value 
is largely sentimental, its worth stems from the fact that 
is has survived many. years with grace and beauty; something, 
very few. people can claim. " 



5, . Question 1 . The following question was distributed to 
all students: 

»'A Founding Father said: *Get what you can, and what you 

get hold; ^ 
»Tis the Stone that will turn 
all your Lead^into Gold.' 
A contemporary writer said: 'If it feels go(^/^o it.' 

.Assignment: What do these two statements say? Explain how 
they are alike and how they are different." 

The following directions for scoring were distributed to all 
readers engaged in the grading of Question 2: 

"6- POINT SCAl/ 
Key to Scoring English Equivalency Essay Examination ^ 

The student is asked to write an essay in which he ex- 
plains what the two statements mean, and how they are alike 
and hovv^ they differ. He should be ^rewarded for what , he does 
weld in his response to the assignment. Papers should be scored 
for their overall quality. 

.An extremely well-,wri tten response may be scored a point 
higher than it would be scored on the basis of content alone. 
A poorly written response may be scored a point lower-.^ 

Spelling errors -should not ordinarily be counted against 
the score. 
POSSIBLE SCORES : 

6 A superior response will be a well-organized essay that 

does the^ three things asked for in the' assignment. It will 
explain the meanings of both quotations and compare and con- 
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trast them; it may explain the meanings t)y means of comparison 
and contrast, or it may explain the meanings and compare and 
contrast them. The ,best essays will note that while the quo^ 
tations share a concern fpr self rather than others, the 
first emphasizes results or (consequences and acquisition of 
achievement, while the second emphasises gratification without 
regard for consequences. The best. essays may well mention the 
figurative meanings in the first quotation or see an objective- 
subjective distinction between them. An essay getting a six^ 
score will show a high degree of competence generally^ though 
it may ^have minor imperfections? It will support generalizations 
with appropriate details or examples., 

5-4 These scores will apply to responses that concentrate more 
on one quotation than on the other, or that deal, with both sub- 
jects somewhat less thoroughly than the essays scoring 6, Essays 
in this group may have minor errors in writing. ^ 

3-2 Papers in this category deal with both quotations but 
may : 

— be lacking in supporting details or examples, or treat 

both quotations superficially; ^ 

— give adequate attention to one but too little to the other; 

— $ail to see similarities in meaning between the two and to 
make distinctions between them; 
' —misunderstand ^or misinterpret the meaning of either or both; 

— be primarily critical 6r argumentative rather than expos- 
itory; 

--have serious faults in writing; ^ 
--drift away from the tppics or reveal considerable irrele- 
vancy. 

1 This score, should be given to any response that is^on the 

topic but has no redeeming qualities. 

Non-response papers and papers that are completely off the 
topic should be given to the table leader. " 
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The following report was prepared by Dr. Rex Burbank, 
question leader for Question #2: 

SUMMARY REPORT ON QUESTION #2 

The two essay questions were devised by subcommittees at our 
first meeting in April, Question 2 was designed to suggest an 
organizing principle upon which the examinee could build a 45- 
minute essay. Two- quotations were used, and students were asked, 
< first, to explain their meanings and, second, to tell how the 
meanings were alike and how they differed. The structure sugges- 
ted was thus based upon explanation or definition,^ comparison, 
and contrast. 

Having decided upon the question, the subcommittee arranged 
with Dr. ^Lid to have it pretested by a group of 25 freshman 
English students at California State University, Northridge, 
under his direction. The subcommittee developed a scoring key 
that set forth agreed-upon standards for grading the pretest 
samples. * 

When the pretest ^.apers came to me several days later, I read 
and scored thenT in accordance with the key arid compared the 
essay scores with the grades given those freshman students on 
the first two essays assigned in their course. There was enough 
of a correlation between scores and grades to suggest that the 
question would generate essays by high school senrors that 
could be scored meaningfully on a 6-point scale. 

We (Dean ^Friedrich, .Dr . Lid, Dr. White and I) met on June 5 
in Northridge, and on that day Dean Friedrich and I read in, 
excess of 100 test papers, written by students who had taken 
the test in May, and selected twenty-five each (he for Question 
1, I for Question 2) for samples. The samples were marked and 
Xeroxed for use at the reading in Pomona from June 16-20. 
During the following week Dean Friedrich and I both attended 
the Advanced Placement readings at Rider College, Princeton, 
New Jersey, he as an observer and I as a participant. In the 
evenings we went over our chosen samples again, analyzed them, 
and wrote our scoring keys to reflect the qualities found in 
'the test papers. The scoring keys proved, with minor exceptions, 
to be both accurate and useful to the readers. A copy of the 
scoring key for Question 2 is appended to this report. 

On Saturday morning, June 16, Dean Friedrich and I met sep- 
arately with the table leaders assignees to us. EaQh of us 
was to supervise the reading of 'responses by fiVe tables of 
, readers, who were to convene in the afternoon. I passed out 
12 sample responses to my five table leaders, and we went 
through each sample, scoring it and discussing the -scores in 
relation to the test papers in order to arrive at agreement as 
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to what qualities or weaknesses would be found in responses 
at all points on the 6-point scale. By noon, the table leaders 
were in close agreement on their grading — their scoring, done 
without prior discussion, was within a point of mine in most 
cases. - ^ ^ 

In the afternoon, the readers met with table leaders and re- 
peated the scoring of samples as we had done it in the morning. 
Though both the table leaders and the readers were inexperienced 
in this type of reading, the 'training session' went well enough 
fpr us to begin the actual reading of 'live' test booklets at 
4 p/m. We continued the next morning but not until after four 
more samples had been read did we resxime reading ^'live' ones. 
Always, the readers were told, the goal was uniformity in scor- 
ing: every student taking, the test, we emphasized, had a rrght 
to feel that his test was ^)eing graded in the same way and by 
the same standards as everyone else'Si^ Readers were asked, again, 
to sacrifice their own grading policies and standards jns^x , those 
established by the scoring key /and by the group as a whoie. 
Most readers understood the need to do this and cooperated. 

Samples were passed* out periodically for the next two days.. 
Readers scored them and their scores »were compared with those 
agreed upon by the table leaders and me. The readers were asked 
to adjust their scoring in^ accordance with the samples. In 
'addition to sampling-, checking was done by having table leaders 
gather papers at random from those already read and scored by 
readers at their table, ' scoring the papers themselves, and re- 
cording both their own scores and those of the readers on a 
•check sheet. The table leaders (who read and scored papers- 
without ''seeing the scores given by the readers) brought the 
checklist with the test booklets to me, and I then read and 
scored them myself without' looking -at the scores given by the 
table leaders and their* respective X'eaders. Thus I was able 
continually to check the scoring of the readers with that of 
the table leaders and both against my own. Where trends de- 
veloped at some tables — such as the tendency to grade too 
high or too low or to settle in the 3-4 range they could be 
and were corrected by passing out samples representing the full 
range of scores in order to remind the readers again to use th^ 
full scale and get clearly in mind once more the qualities of 
papers at each point in the scale. A high degree of agreement 
was achieved, and in the va'st majority of cases the two readings 
of Question 2 were within one point of each other. When there 
was a spread of two points a response was read a third time by 
an individual in a special group of our best readers chosen for 
this task. Papers (and there were remarkable few) with a spread 
of 3 points were given two readings by that group. 'Most papers 
with a spread of 3 points were radically uneven in quality and 
so the discrepancies were understandable. I'm satisfied, however, 
that generally the papers were graded with a very high degree 
of uniformity, reliability, and validity. 
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We didn't expect perfection from even the best of the stuaent 
responses, but for a score pf 6 we required that the paper show 
a high degree of writing skill in doing what was ^ asked by, the 
assignment. The essay was expected to say something, d4.rectly 
or indirectly, about the meanings of the quotations and com- 
pare and contrast- them. The best papers were • expected to demon- 
strate the ability to build on the implied organization yith 
nieaningful d^etails, to develop the general, points the writer 
made, and show a high degree of understanding of the quotations.. 
The best papers, for instance, revealed a consciousness that the 
first quotation looked to the future, to consequences or re- 
sults, while the second emphasized immediate gratification of 
desires or wishes; the first was figurative and stressed acquis- 
ition, the second was literal and suggested personal satis- 
faction. Students writing the best papers accurately perceived 
one -or more such distinctions.^ 

A paper given a score of 5 fell just short of the 6 essay 
in having- minor faults in writing, being slightly less well 
developed, or displaying a slightly less acute understanding 
of one of the quotatians; or emphasizing one quotation over y 
the other. A paper given a; 4 differed only in degree from those / 
given a 5; it was awarded a 4 rather than a 3 in that, overall, 
it suggested competence, despite whatever minor faults it might 
have, rather than incompetence. 

We asked the readers to decide first of all, as they read, 
whether the paper was an upper or a lower half (that is, a 
6-5-4 or a 3-2-1 )essay, to look at the quality of the paper 
as a whole, first, and then to make the necessary distinctions 
within those two categories. We asked them to forget the letter- 
grading they are used to in their own teaching and to remind 
themselves that it was essential to use both ends of the scale 
as well as the middle; only then could we make relative judge- 
ments and pertinent distinctions. 

Papers given a 3 grade were lacking in details, or gave too 
little attention to one of the quotations, or failed to per- 
ceive similarities in the quotations, were primarily critical^ 
rather than explanatory, lacked unity, or had serious faults in 
writing. A 2 paper, had one or more of these weaknesses in greater 
- degree than one given a 3. A paper "was to be given a 1 if it 
was on the topic but was so badly written that it suggested 
illiteracy or clear incompetence. We gave a 0 to off-topic 
essays or papers with no response at all* Examples of papers 
given ^scores 1 to 6 are appended to this report, as are the 
forms used in the readings." 



^The following student responses to Question #2 were sample 
papers used during, the reading to illustrate the grades on the 
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G-point scale: 



' . ' SCORE OF ONE , ' 

> * ► * * 

"Get what you can, and what you get hold'; tis the stone 
that will turn all your lead into Gold, 

This "Statement attributes ,to the young, while working 
-hard in life one may establi-sh a foundation and from this point 
moiitier this foundation into a future whi'Ch in. later years will 
prosper with prosperity and fortune," 



SCORE ,0i^ TWt) 

"These twO' statements are from two widely separated times 
,the first was spoken in a time when life was short and hard 
"when most peoples were poor and the wilderness was sometimes 
just behound your door, / . " . 

The other was writen at a time when just the reverces was 
true life' is long- and for the most part easy when most are pros- 
press and to find adventure one must go looking for it rathir 
theri*^ having it^ come chasening after you. 

As they come from two diffrent ages they represent two 
diffrent views of happyness the first is a life time of happi- 
ness not only for you but for your kids. The other is a day to 
"^day kind it seems to lack the resposably of the other, it does 
neb say anything abo,ut' the resultes on others the first state- 
ment does not say this th'ings eather but the implication is 
there." It depends very much on you point of view.' Thou in 
maney ways that dif fure in the end- thay both mean the same to . 
be someone 'you must have something only in .this way will all 
the troubles of you life have meaning and be worth it and all 
your lead be turned into Gold," > 



SCORE *0F THREE 



"The practical aspects of human motivations should not be 
confused 'With immorality. The Founding -Father offered sound 
advise. Paraphrased it means: One may attain his realistic 
goals by diligently working at one's maximum level. The con- 
temporary author says: . One should wear his morals like a suit 
of clothes; whatever comfortable is fine, only they should not 
tight under the armpit§,. 

No basis of comparison, I believe, can find common philo- 
sophical points between the two statements, I, therefore, find 
the question a loaded one. Only if one were to misunderstand the 
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true intentions of the Founding Father could there be a likeness 
comparison, fore the contemporary author's statement is easily 
understood, and not apt to be confused. 

Pe'rhaps analysis of the statement is indicative, 

•Get what you can, and what you get hold. ' 

' 'Tis the Stone that will turn all your. lead into Gold.* 

Today, frequently, English speaking people corxfuse the 
meanings of words and generally speak poorly. The word/can' • 
in the first line is a modal auxiliary, and when used as such 
means 'ability'. If the line were to contain 'may,' then. the 
advise would not be refering- to a goal that could be acheived, 
but rather to 'get what you may '^ — whicfi coanotes immorality 
of dishonesty. 'Get what you can' therefore means, 'get what 
you have the ability to get. Although the. 'can' could Connote 
immorality it probably does not, because the erudite Founding 
Father undoubtedly knew the rhetorical difSerence between 'can^'* 
and 'may' and therefore did not make a mistake in syntax. 

The second line: 

•Tis the Stone that will turn all. your Lead intp Gold,' 
refers to the Philosopher's Stone gf Alchemy. This, seemingly 
would' role out the idea that the Founding Father's message is 
concerning morality, or anything neccessarily intangible. It 
s.eems rather obvious that the what one can get and hoAd must be 
material possessions. In this aspect, theri,^ the statement by 
the contemporary author has little relation at- all to the 
Founding Father's statement! ^ , ' • - 

The statements have little relation to each pther. To say 
that they do would mean that anything practical is immoral." 



SCORE OF FOUR 




"The first quote says to- find all opportunities, and when 
you find them, keep them because they, will bring fortune ±q you^ 

The second quote says to do the things that, make you feel, 
"good" csr happy. Things that jnake you feel worthwhile are what 
you want to do. 

The two statements are Similar because they both want the 
reader to get ahead and find happiness. The first wants you to 
find success, and through success, richness (joy or happiness). 
The second warlts you to find joy and happiness in doing some- 
thing that comes natural to you. 

The two. statements have differences, not only in^their 
words, but in their ways of obtaining 'happiness. The first tells 
the reader 'Get what you can' but doesn.'t put restrictions on ^ 
how to get it. In the second verse of the first statement 'will ' 
turn all :your Lead into Gold' could be taken b}^ the reader to 
mean that^ all /Dppor;t:unities will lead to good while the aliithor; 
in general reading, * seems only to be saying always strive ^^r 
better and don't let an opportunity pass you because of in- 
difference or Indecision. The second statement cXould^alsd have 
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a bad connotation ► for some r^eaders, such as critics, because it 
could be taken to' mean do something even if it is wrong, simply 
because it makes you happy. 

In general reading', though, the first statement would 
seem to be an enionged version of saying the same thing as, the 
second statement." * « _ 



SCORE OF FIVE 



"The two statements, one .by the Founding Father and one by 
a contemporary writer, are very similar, and also very diffetemt. 

The basic thought in both of these is to get or do what 
you want. If you have, a goal,, or an object that you want to 
obtain, push and shove, or squirm and wriggle, but work towards* 
your goal ^to get it. This is one of man's basic impulses — the 
writers are just encouraging the fylflllment of man's wishes. 

The thought in the statement made by the Founding Father is: 
the foremost example of the idea of getting an object. It seems 
that the writer^ urges man to acquire materialistic things such as 
land, wealth, or possessions. He" urges you to get it; and once 
you get it, to , keep it. 

The Contemporary writer seemed to be speaking more of 
spiritual or nonmaterialistic things. He seems to urge that man 
expr^ess himself freely in his speech, his actions, his ;life style 
J^ove can be more easily meant in this statement than irf the one 
hy the Founding Father., This contemporary statement is used 
.quite often in regards to the /New Morality. ' No^ one really knows 
what that is, but everyone, wants to .be* part of it, and the 
phrase, /If it feels good, do it», has bec9me a sort of .cliche. 

The Founding Father and the contemporaji^y writ;er both seemed 
to want to express the idea that man must look after his own in- 
terests, be they pleasures or businesses. 

Whether or not the two writers had any of the'' same -ideas in 
making their statements is not known, but the similarities in- 
dicate that it is very possible. I'm sure the thoughts of the , 
Founding Father were more conservative than thpse- of the con- 
tempo r^ary writer. 

These two statements both say a lot about man's way of 
life, now and in the past. The acquisition of materia,! istic and. 
spiritualistic pleasures has always been a principle part of 
our lif^e. " 
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"People today are looking at life in an entirely different 
perspective than those who witnessed the birth of this nation. 
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Life is no long^x a cons.tant struggle for survival (as it was * 
then), but instead iT: is a^time for people to expaijd and. grow ip . 
knowledge and understanding* Today, people "live- tjieir lives by V 
an entirely different 'philosophy than, our . ancestors* of yester- ' ^ 
year. * ' «. 

A Founding Father once said, 'Get what you can, and what 
you get hold; 'Tis the Stone that will turn all your Lead ifito 
Gold. ' The philosoph/ he is expounding is that of being a 'go- 
getter, ' a person who can (^et all he cani But it is, also a 
philosophy of values. This means that one should also val^^^ 
what he ac^ieVes or gets, and never le± go of it. In thbjse days 
things *of value were often hard to come by, so they valued things 
greatly. Because it might be that ^ one little item that could 
'turn all your Lead tV.Gold,' or make you prosperous*' It is a 
liberal philosophy in/Eftat it says to go out and ^get whatever 
you can, yet it is conservative in that it ^ays to hang on to 
it and not to let go* It ,^'s a philosophy of hard work, and being 
appreciative of that work. 

^6ut the geople of -today's society have great difficulty-* 
following a philosophy of this nature. We aren't living in 
'hard times', at least the majority' of Americans aren't, so it 
is hard to follow a philosoph*y' thaf-old, or seemingly dated* 

A contempo'rary writer states, 'If it,^feels goocf^l^o it,' 
and it appears as though people are 'doing it', because we are 
living in a completely dif f er^nt^vay .that our Founding Fathers 
did. People today don't feel the need to hang on to things in 
hopes of it producing great prosperity* L'ife is too 'easy going' 
and the struggle for survival is* non-existant* So people decide 
what they want to^do, and they^ do it. There are usually no great 
risks .involved (as there were earliei:) and in gener.al , life has 
turned into more of a game or \r<jutine> for people than a^struggle 
or a challenge. ^ ' » - 

The two philosophies do have'' some, common ground. They both 
say to go out and do, act, achieve,, get, accomplish. They both 
call for action, but one says to hang oh to the dividends and 
the latter of the two says to move on to .something else , that 
' feels good. ' * 

I feel the latter of the two philosophies shows the real 
moral deterioration of our society today. Life has become so 
routine, so mechanical , so dehunjanized that it seems ^^to be 
almost lifeless. And so a contemporary writer tells people to ^ 
do what feels good. What feels good is not always what is best, 
and often times we have to do things that don't feel, good, but 
they must get done. If one -always does /.what . feels good, a lot 
of problems go unsolved and new ones are* created. ^ 

I favor the first philosophy because it pu-ts 'value in what 
we do and it puts purpose in life: that purpose .being wdrking 
hard and appriciating the end result* it may not always feel good, ^ 
but will provide self-satisfaction in knowing that something had 
tq 'get done and you did it.*' > 
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6. The' Nbxm Sample . One of the central problems in the 
test program was the establishment of norm scores.. The ob- 
jective test had been' normed in"1966, on a group of 
Students from 17 schools. It was not altogether clear that 
that norm sample was in fact representative' of- the State ^ 
University population, although recent experience with the 
CLEP Subject Examination, Analysis and Interpretatibn of • 
Literature, at the University of Texas has validated the 
norm scores, rdeally, .a new norm sample taken from Calif-^ 
ornia State University and Colleges students completing -a 
year of freshman English would have supplied new norms for 
both* parts of the examination. It proved impossible to find 
the time and money to provide such a norm sample for the 
objective test, and we decided to stand with the norn\ pop- 
ulation achieved by"" the CLEP program in 1966 for the objee- 
tiv'e test.' However , we did seek to obtain a norm sample 
from California State^'hiversity and Colleges students for 
the essay test. 

The results of our attempt to acguir.e this norm sample 
were disappointing and not really reliable: we had been » 
promised almost 600 test papers from 12 different institu- 
tions; we in fact obtained 259 papers from 7 instituti^bns , 
* and almost half of the papers represented the State Poly- 
technic Universities of Pomona and San Luis Obispo. It also 
became clear that the motivation of the students writing 
the norm sample at the end of their class work was far be- 
low that of the students taking the test ,for credit. 
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The norm sample did ,show, however, a correl^ation be- 

tween scores achieved on the -essay and projected student 

•grade in the course, and, hence, tended to validate the / 

accuracy of the essay grading. If the norm sample were to 

be considered valid, it could be used to show that the 

students given credit through the testing program -generally 

write better than the students receiving credit through 

freshman composition cLasses. But that,^ would be a risky 

conclusion to draw in the light of the project's failure 

to accumulate a sufficiently representative and well mo- 

» 

tivated norm sample on -the essay portion ^of the test* (See 
» ' 

Table I and Table II, pages 56 and '57*) 

'Future attempts to gather a norm sample for this pur- 
pose will need to take into account the problems we optim- 
istically felt we could solve. We made personal contact on 
each of the 12 campuses with directoi^s of two-term freshman 
English courses, and each department chairman had agreed to' 
participate in the norming projecto We accompahied the re- 
quest for participation with the letter which follows and a 
packing sheet (see pages 69 and 70) in order to assist the in< 
structors in giving both the anticipated course grade 
and a forced ranking of his students (in case he chad theoret- 
ical difficulties with the conventional gradjLng system). It 

p 

was our expectation that the personal contact, the general 
willingness of English ''depar'tments throughout the system to 
cooperate with the project and the very specific instructions 
contained with the norming materiaj-s would all lead to a 
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CALIFORNIA S T.A T E U N I V E'R S I T Y ,^ NOR f H*R I d G c. 

Northridge, California 91324 

lor 

dlpaktmcNTO^" K%cu$M . May 14, 19V3 




D^ar C3UC English Professor: 

We appreci^ate your willingness to participate in the norming of t,he 
English Equivalency Test, which was administered to over 4,400'prospective 
entering .students on May 12, 1973.. The norm sample that you will help 
collect is absolutely crucial to the proper grading of the essay tests, andv 
w?/arc asking you to read and follow the instructions on this sheet with 
^ great care. 

^ The only essays we can use are essays .^^nritten. by students completing 
^'^vo terms of freshman English. Please tell the students talking thts c:<n{n- 
inati'On that the grades on the essays will be part of their course grades 
and please do all you can to encourage them to do thoir best writing. 
But if you intend to make any inarks on the papers or to return them to the 
students, please make a Xerox for such use. We must re ceive the c^>nay hook-- 
lots v nrrarked in order to be able to use them . Allow th-j students e::actly" 
. 45 nlnotes for each essay. Be sure not to open the essay questions until 
you administer them. All answers must appeai; in the enclosed green CLEP 
Essay An5_wer Books, and no student should need more than one book ^or botli 
essays (thus you need to redistribute the green books if you use two separate 
class hours.) if necessary, a second book may be used. 

^ Thereover sheet of: the answer book must be indistinguishable from the high 
school books from the Hay 12 test, if we are to use your' papers; thus, ask your 
students to use their hone address (not a doraitory), to fill in for Subject 
"English Test," and to insort your college or university nama for Center Name. 
If a student does not know his social security number, he may \eave that item 
blank. Above all, have the studeiits enter May 12. 1973 for date, no matter w hat 
date you, nay in fact give the test . ~~ ~ ' '. 

. .We are also asking you to give us soine essential additional information 
about each student. Please, fill in the enclosed packing sheet carafully, for 
without cnat inform.3tion the papers you send us cannot be used. The lant column 
'Co:nparative Ranking" needs some explanation. Choose the three or four weakest 
students in the class, and write "1" for their ranking; choose the thrcc~W 
best students and call them "6." The average students should be split between 

iro.n 1-5 o^ all your students, without regard' to grades. 
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It seems clear from the nationwide reaction to our test prograra that 
the success (or lack o£ success) oE what we do will have a prolound irapact on 
national testing programs and on the^ future o£ essay testing in such prosrarns, 
As far as know, this is the first time English professors have assiT..cd 
responsibility for the reliability and total administration of such a test, 
and the program thus becomes an example of professional activity v/ithout 
precedent, the scrupulousness and fairness of 'our procedures will be subject 
to (Sareful scrutiny, and your p^rt in these procedures is ceiihral to the 
entire grading process. 

Return the entire packet not later than June 5 to Professor Richard Lid, 
Department of English, California State University, N^orthridge, Northridgc, 
California 91324. ' • . . 

Sincerely, 




Edward Wiiito, Director ' 
California State University English 
Equivalency Trtst 



Enclosures 
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' CSUC ENGLISH EQUIVALENCY "TEST 
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Packing Sheet for Norm Sample 

"^Faculty Member Course Number 
College or University . Number of Tests 

• 

Student Name Social Estimated or Comparative Ranking 
(Alphabetized) Security i^nal Course (see cover letter)^ 
. " Number - Grade (not an (use 6-poiht scale) 

examination grade)^ 
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^xeliable, useable, and heavy returno As Table I shows, 
''these expectations were not well foundedo The rather poor 
participation, even by English departments who had agreed 
to participate in a test they themselves were sponsoring, 
is very clear evidence of the residual resentment most 
English departments continue to feel towards equivalency 

m 

testing. 

University of California Norm Sample p Two sections 
of students .completing two terms of , freshman "English at 
the University of California wrote the essay test, and 
their papers were graded during the essay reading sessiono 
As with the California State University and- Colleges norm 
sample, the graders could not distinguish these test papers 
from the others; the University of California sample was 
coded and later separated. 

Twenty-four students from University of California Los. 
Ang^^les received a mean score of 14.29 (standard deviation 
3.54); .16 students from University of California Riverside 
received a mearl score of 13^56 (standard deviation 3.31). 
The total University of California sample of 40 students 
achieved a mean score of 14.0 '(standard deviation 3.43), 
coihcident^lly the- passing score' set for the test group. 
7. The Essay Reading . The experience of the Advanced 
Placement program based on substantial ETS research on 
essay grading, has shown that for an essay reading to be 
reliable, it must take place under controlled conditions. 
Thus, aja early decision was made to organize an essay reading 

-'64 
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'session of approximately one week", during which essay ' ^ 

graders would be gathered together and work together in 

order to achieve group grading standards which would be 

fair to the students who had written the essay test. 

After a survey of local accomodations, it was decided to 

make reservations at the ""Kellogg-West Conference center, 

at California State Polytechnic University, Pomona for the 

week of June 16-20, 1973. That conference center . supplied 

accomodations for about 75 staying at the center, working 

space (tVo large rooms for the ten tables of readers; and 

three smaller, rooms for supplies, statistical work and the 

posting of grades), and excellent food within the state- 
t 

allowed pier diem for college professors. The date was 
chosen in order to allow ETS sufficient time to score the 
ub'jig'Ctxve test and return completed information on the 
scores to the test administrators for use during the essay 
grading session. (ETS also supplied two complete sets of 
mailing labels, so that notification of the results could 
be mailed to the students speedily.) 

The professional compe1:ence o^ those reading the , 
essays is crucially important to the reliability of an 
essay reading. Since the project was sponsored by the Cal- 
ifornia State University and Colleges English Council, the 
project directors requested each of the nineteen English 
Department chairmen to nominate from the full-time English 
Department staff at his campus those who would be both 
willing and able to participate in group essay reading at 

65 
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the time scheduled. Since the English Department chairmen 
nominated the readers, and screened them for their ability 
to participate, it was a relatively simple task to appoint 
readers. The care with which these nominations were made 
is reflected in the reliability statistics of the reading, 
among the highest ever achieved. Seventy-five readers were 
appointed, and, of these, the ten who were the most highly 
recommended, or who had had previous successful experience 
with controlled essay reading, were appointed as table ^ 
leaders responsible for insuring *the reliability of reading 
at one table. The/question leaders {S4e pages 17-18) 

coordinated and Supervised the work of ^the five table 

* * J 

leaders in each room, and directed -the discussions of^ grad- 
ing criteria. 

It was difficult to estimate how many readers -should 
~ ' be ""appointed. Not until the end of April was it khoWn how^ 
many examinations would be graded, apd there was no way of 
knowing the speed at which the reading would progress. We 
estimated that* a single reader would be able to read and 
score ten papers an hour, and that an eight hour work day 
. would allow for seven full hours of reading with time for* 
* rest and discussion of standards. We, thus, estimated a 
single reader could rfead and score 70 papers a day, or 280 
during the four days of the reading. As it turned out, we 
underestimated the capabilities of our highly professional 
readers, who, despite their lack of experience, in fact 
were able to read and score 120 papers per day with, great 

ER?c . • 
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accuracy. 

In order to insure fair reading of the papers, several 

♦ 

steps were taken in preparation: a) Randomizing / to insure 

that ho single »batch* of papers would come -from the^ Same 

location. ' Essay ^booklets from the nineteen test centers 

were randomly arranged intp batches of 25. Each batch of 25 

was given a batch number and batch cover sheet to insure that 

it moved properly through the reading. b) Labeling . Special 

/ 

I'abels were given to us by the ETS 'office in Berkeley. The 
labels had f our'^detachable squares which would allow scores 
to be recorded through carbon on the test booklet, and yet ^ 
be concealecJ ''from subsequent readers. At the conclusion of 
the fourth independent reading, the carbon paper ba^cking was 
torn* off and all four scores were exposed to be tabulatedl 
c) Coding . All papers from the norm sample were coded on 
"the back cover so that they could be separated from the test 
papers and given separate statistical treatment. The, norm 
papers were then inserted into odd numbered batches from 1 
through 55 so that they would be easily separable from the, 
others. . • 

At the start of the reading, one reader aide (a local, ^ 
commuting, student assistant) was assigned to each table. 
One was assigned to each of the chief aides, and one assigned 
to the statistician. After the first -day, it became clear 
that ^additional student aides were needed, and through' the 
office of the English Department chairman at Cal-Poly Pomona, 
additional -pnames were provided to us. Thus, in addition to 
the original thirteen aides, at the peak of the reading an 
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additional six aides were being used, along with a^ few 
readers who were assigned to work with the readers'^ aides 

I 

in the posting of grades • 

When the four readings of the essays ,had occurred, and 
the scores wer,e revealed, there w.^ire occasional booklets on 
which large differences (2 points or mC)re) appeared in the 
grading of one, or both; of .the essays. These papers were 
'referred back to the question leader, ^who either reconciled 
the two grades' himself, or assigned the paper, to an -exper- 
ienced reader for reconciliation. An additional group ^of 
papers, from students with particular 1^^; high oJ:)jective test 
scores, but low essay scores, was also referred to the ques-- 
tion^ leader^ for reconsideration, and a few of them were 
fescored as passing. ^ / 

The management of the essay reading is a huge under- 
taking, requiring a" substantial amount of time and attention 
if it is to go well. A staff of approximately .100 individual 
was .employed to conduct the reading at the conference -certter,, 
and on the whole the reading went effj.ciently and success- ^ 
fully. The project directors did -underestimate the amount 
of labor and coordination involved in running the *essay 
reading properly. But, in part due to the efforts and coop- 
eration of the management of the conference center, rhe 
reading was concluded successfully. 

E, Converting the Es'say Grade and' Combining the Scores. 

The objective test is scored on the familiar ETS scale 
which converts the raw score to a scale ranging from 20 
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(lo.w) to 80 (high) with a mean of 50 and a standard devia- ' 

tion of "10. Since we decided to weight the essay portion 

of the "test equally with the objective portion, it became 

necessary to convert the essay scale (which ranged from 

4-low to 24-high)yto the same scale as th"^ CLpP scale so 

that th? scores could be combined. After a prolonged period 

of consultation with statistical experts, it was decided to 

accomplish this score 

method. The following 

the procedure used: / 

"'Equating* scores means making them equivalent 
with respect to some characteristic. Here, the essay 
scores were^ transformed to the same scale (20 to 80) 
as the objective scores so that the transformed Qssay 
scores could be added to the objective scores to ob- / 
tain a composite score in which the two parts would / i 
be equally weighted. Had the raw essay scores been / 
added to the objective scores, the essay part would 
have had a weight of only 1/4 of the tota^. Also, the 
transformation was. done in such a way that the con- 
verted essay scores can be directly compared with 
objective scores. That is, a converted essay score of 
48, for example, represents the same level of per- 
formance a^'s an objective score of 48. 

"Of several acceptable methods of equating 
which are available^, the equipercentile (curvilinear) 
, method was selected as the most appropriate. The con- 
verted score corresponding to each raw essay score 
is that score in the distribution of objective (CLEP) 
scores such that the percent' of the group scoring 
below that objective -score is equal to the percent 
of the group scoring below the given essay score. For 
example , to find the converted score corresponding 
to a raw essay score of 15, we observe from the dis- 
tribution of essay scores that about 68 percent of 



1. Angoff, W.H.; JScales, Norms, and Equivalent Scores'* 
in R. L. Thorndike (Ed. ) Educational Measurement , 
2nd Ed., Washington, D.C. : American Council on Ed- 
ucation, 1971, 563-590. 



conversion by the equal percentile 
statement by William Cowell summarires 
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the group scored below 15 on the essay part. From 
the distribution of objective scores it is determined 
that a score of 53 is that score below which 68 per- 
cent of the group scored. So a raw score of 15 is 
equivalent to a scaled score of 53. Similarly, a 
scaled score is found for each raw essay score. Table 
III vshows the converted score corresponding to each 

raw essay score. 

f « 

o 

'•V. Technical Notes about the Equating . 

(1) The equating was done using data from the Equa- 
ting sample .... The sample was used instead of the 
total group so that "the statistical work could pro- 
ceed along. with the essay reading. Score conversions 
and^distributions could be prepared for the sample 
data in time ^for a meeting with Quests^on Leaders at 
the reading. " The" time and cost of equating ^was^ also 
reduced by using a sample rather than the total group. 
Scrambling the essay booklets -before the reading , 
provided a* sample which was representative of the 
total group. ^ % 

(2) A method of graphical2smooting, described in 
•Educational Measuremignt ' , was used to interpolate 
between perceni:iie ranks in the distribution of ob- 
jective scores. 

(3) Although the fequipercentile method was selected 
because it allows for a non-linear relationship be- 
tween objective and essay scores, the correlation 
between raw essay and converted essay scores is .999. 
This ex^tremely high' coe:fficient of linear relation- 
ship indicates that the simpler linear methods of 
equating Qould have been used with essentially the 
same results as the equipercentile method. 



2. Ibid . , pa^e 572. 
,^ 3. Ibid . , page 513. 

See the Appendix for a series of statistical reports 
prepared by Dr. Cowell on the project, including a full 
discussion of the problem of equating essay and obje^ctive 
test scores. 
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TABLE III. 
' • . •/ spORE 'conversion TABLE 

• gSUC ."ENGLISH "EQUIVALENCY TEST ' 
/ ^ ' MAY .1973 N 

* Essay Total Score - Converted Score 



24 


- 76 


23 


- 73 


22 


- 71 


21 


- 69 


70 
^ \j 


-' 67 ■ 




- 64 




- 62 


17 


- 59 


16 


- S6 


15 


- 53 


14 


- 51 


13 


- 48 
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- 46 ' 


11 


- 43 
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10 


- 41 


9 


- 38 


8 


- 35 


7 


- 33 
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T 31 


5 


- 28 


4 


- 26 


3 


- 24 


2 


- .22 


1 


- 20 


0 


- 18 
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F. Determination of the Passing Score. 

One of the most difficult decisions wafe to establish 

the passing score on the essay test. One possible passing 

^score could have been d total of 16, that is, a paper ivha^ch 

was given a score of 4 on the 5-point scale by each of the 

t i 

4 readers. However, the 16 converted to a score of 56 on the 
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20 to 80 spale and clearly represented a level of perfor- 
mance above that which freshman English professors norm- 
ally e^xpect of their students at the- end of the course. Pn 
the other hand, it would have been possible to set the 
p,assing score at 11, since the "C" students' in the norm 
sample (see p. 68) were obtaining essay ratings of about 
that level. Once again,^ the conversion of the 11 to the 
CLEP scale (43) demonstrated that that score was clearly 
too low and that the grade of "C" reported for these norm 
sample papers was not an accurate reflection of their 
ability. 

After much 'consultation and discussion, the follow- 
ing committee agreed unanimously that we should set the 
passing level at a score of 14, and that a minimum pass- 
ing level of 13 should also be set: Professors White, Lid, 
Burbank, Firiedrich; ^Test Officers Abbott, Cantey, Bradley 
(from Cal-Poly, Pomona); Statistician William Cowell. 

The arguments for a passing score of 14 were as 'follows: 

1) that score represented an average (jrade of two 4's' and 
two 3's; that is, two of the college pr^ofessors grading the 
essay felt that, the essay to be passing at the lov;est level., 
and two felt it to be failing at the very highest level of 
failure, 

2) the score of 14 converted to 51 on the 20 through 80 \ 
scale which was very close to the mean '"C" score on t^he 
objective test of 49. I 

3) the norm sample, in so *f ar as it could *be relied upon. 
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4 

showed that the mean score for students receiving A, or 
C was 11.9, and, when awarding college credit, it is well 
to err a Kit on the side of severity. 

4) the mean score of the University of California students 
completing a year's work in freshman English, a sample 
apparently better motivated and better administered than 
our own norm sample, turned out to be precisely 14 on a 
group of 40 papers. 

The same arguments were used to justify a minimum 
passing score of 13, which converted to 48. A student could 
not pass the test with an essay sc^ore below 13, and if he 
or she had a score of 13, an objective score of 52 or above 
(which was achieved by 32 percent of the test group) was 
necessary for a total passing score. 
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•/ G. Notification of Results . Each student who took the test 
received a score report from ETS giving the results of the 
objective test. An insert was included in that notification, 

4 

4 

pointing out that an additional report, which would give the 

/ 

results of the entire examination (including the essay test) 
would follow. 

The administrators of the .test program prepared two 
letters, which were sent to all students. The intention of 
the* letters was to stress the meaning of the examination, 

A 

to test college-level ability, and, particularly^ to stress 

tliatjj those who did not pa^s- should not think of themselves 

as "failures." 

> 

The following letter was sent to those who did 
pass: ^ ^ 

"Dear 

On behalf of the English faculties in the California 
State University and Colleges, we congratulate you for 
successfully passing the Freshman English Equivalency 
Test you took May 12, 1973. , 

You are among a select number of students who have 
demonstrated the same level of ability ^any students 
achieve J^nly after a full year's work in college English. 
In particular, you have shown that you have college level 
competence in writing es3ays and reading literature a 
competence we urge yoii to develop.^y taking the more 
advanced English classes you are now entitled to enter. 
, The 'test you passed consisted of the ninety-minute 
subject examination. Analysis and Interpretation of 
Literature ( part of the College-Level Examination Pro- 
gram of the Educational Testing Service), and* a ninecy- 
minute essay test composed and graded by State University 
English professors. In order to pass, you needed to 
perform satisfactorily on both parts, and achieve a com- 
bined -score equal or superior to that of the average "C" 
student who has completed a year's work in Freshman 
English. 

You should take this letter with you when you regis- 
ter for classes at any of the nineteen California State 
University or College campuses; it will allow you to 
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claim six semester units of college credit for what you 
know and' permit you to enroll for courses with Freshman 
^English as prerequisite. You may want to consult with 
the English department chairman about which English 
courses will be most appropriate for you. 

If you do not plan to enroll in the California State 
University or Colleges, you should still save this 
letter, for it is the only notification of your success 
that we can send you. Institutions outside the State 
University system may well give full or partial credit 
for your passing score, and you should inquire about 
your chances of .gaining credit. 

Please be sure to complete the enclosed information 
card and return it immediately *so tha<t our records can 
be complete . " 

The following letter was^sent to those who did not 



pass ; 



*Dear 



Even though •you were not among the group of Students 
who passed the Freshman English Equivalency Test., May 12, 
1973, you should not consider that yqu "failed" -the 
test. Your level of achievement on the test did not 
match that of the average college student who has com- 
pleted a year's worlc of college English; those who 
passed the test were able to accomplish on their- own 
what everyone else does in a year of college ^class 
work. Only those who combine natural ability at English 
with careful and systematic college level study have a 
' good chance at such college equivalency credit, and it 
is no surprise^ that many •^able high school students did 
not pass. 

Remember, the test v;as not an aptitude test, measuring 
your native abilities at English; it was an equi<;'alency ' 
test, measuring your Tevel of achievement.^ Thus, you 
may be very talented at reading 'and writing — you pro- 
bably are, since you elected to take this test — but 
the test results showed that you have not yet developed 
these talents* at the college level. 

Your total test score will not be sent to anyone or 
any institution, even on request, and the only record 
that you took the essay test will be kept for research 
. purposes at the English Test Center. (Your score on the 
objective portion of the test may be sent by the Educa- 
tional Testing Service, only on your request, wherever 
you wish. ) 

When you return the enclosed information card, you 
will make it possible for us to do this research; we 
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want to know more than we do about how to help able and 
ambitious students prepare themselves in English, and 
we need to evaluate the results of the 1973 test in 
order to prepare better ones in the future. We apprec- 
iate your help in this research, and we urge you to can-- 
tinue to follow your own interest in writing and "liter- 
ature as you go through college," 

During the summer, the office of Institutional Research, 
in the Chancellor ' s ' Of f ice , prepared a roster of those who 
passed. This of ficial^ roster was sent to each California 
State University and Colleges admissions office,' and was 
used to corroborate stu(^ent applications fojf credit on indivi- 
dual campuses. In September, after the University of Calif- 
ornia had decided to award credit on the sante basds as the 
California State University and Colleges system, one additional 
roster was sent to Berkeley to^ be used by the University of 
California system. 

A few requests for info^rmation about the examination 

. have coinf from private institutions., whose students have 

* * 

passed the examination and sought credit'. These requests 
have been dealt with on an individual basis. In addition, a 
number of secondary schools have requested information about 
their students* scores. If the program expands, the handling 
of test results will need to be reconsidered, since ^lese 
problems will surely grow. 
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H, Public Information. 

"Whai: if they gave a war" the bumper sticker says, 
"and nobody came?" What if ^ gave a test and nobody came? 
Throughout the development of the test program we were 
haunted by the fear that the program, so expensive, so 
painstakingly developed, so carefully j)repared, would not 
attract enough students to make it worth the ^giving. Cer- 
tainly high school students, particularly the most able 
ones, are subjected to an extensive series of tests in 
preparation for college: they take the various college 
board tests Mvanced Placement tests various aptitude 
tests, and, of course, their vari'ous final examinations in 
their classes. Hov were we to draw the attention of these 
able high school students to yet another test program,, and 
motivate them to spend $15 to take this new kind of test? 

it thus became clear by the middle of January, 197?, 
that a large public information effort would be necessary, 
if the students who stood to profit from the English 
Equivalency Examination were to enroll for it. We planned 
a two-month campaign in* March and April to attract the 
attention of students to the new opportunity for college 
credit we were offering them. This campaign encompassed 
the following activities: 

1 . Each high school counselor listed on the Chan- 

cellor's office list received 5 registration forms 

and one poster in the mail. 



2. Each member of the California Association of 

Teachers of English received one registration f orm ^ 

and one poster i-n the mai,l« 

K 

3. The Counselor's Diges t issues of February 1973 
and April 1973, featured major articles on the front 
p^ge about the English Equivalency Examination. Every 
high school counselor received thes^e issues. 

4. Each of the nineteen California State University 
and Colleges Admissions -Offices was requested to include 
one registration- form for the test along with infor- 
mation to admitted students during, the month of March 

and April. 

t. 

5. The Office of Public Relations, in the~"Chanceri6r ' s 
Office prepared several press releasees which were dis- 
tributed to newspapers, radio and television stations 
around the state. These releases led to a major article 
in the Los Angeles Times , and widespread newspaper, 
radio and television coverage of the announcement of 
the test in early March. The director' of the program 
was interviewed on KNX New Radio in Los Angeles, and — 
television announcements were made in the San Francisco 
Bay Area and in San Diego. Local campus news relea'ses 
foJLlowed, in many cases, and 3 series of announcements 
dppeared in various media during March and April. 

6. A few of the California State University and Col- 
leges English department chaj^^rmen took special pains 
to notify high schools in their vicinity of the hew 
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opportunity for gaining college credit, and th^se 
chairmen sent registration forms to high school English 
department chairmen with personal notes attached. 
This public information campaign was costly and time- 
consuming, but absolutely essential to the success of the 
project, * The number of registrants for the examination, over 
4,000, was beyond our expectations and ample evidence that 
the program met a genuine 'neted; many of those who could pro- 
fit from the program found out about it. Nonetheless, during 
the month of April and in subsequ^ent months, we kept hear- 
ing of students or entire high school classes w,ho knew 

nothing about the- test program and were disappointed at not 

i 

being able to participate in it. The public information 
campaign did not reach all who should have heard about the 
examination, and publicity efforts should be improved upon 
*in the future. The weakest point in the diss^emination of 
information ^'bout the test seems to have beei;i the farlure * 

of high* school counselors to communicate information to 

I 

high school English teachers, who, in turn, ^would encourage 
thei^best students to enroll for the examination^ Perhaps 
articles in high school student newspapers would solve 
this particular problem. 

Nonetheless, an extraordinary amount of publi<^ity was 
generated for tfie test program in a very short amount of 
time. If the program continues., it should become generally 
known, and a regular part of the spring testing cycle for 
high school seniors. - * 
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Then efforts will need to be made to attract adults, 
very few of whom took the examination, 

IV - 

V 

FOLLOW-UP STUDIES 

The Office of Institutional Research in the Chancellor's 
* »■ 

Office, has put all information about the students tested on 
computer tape, and has assumed responsibility for conducting 
follow-up studies in order to determine the effects of the 
examination upon the careers of those students -who took it 
and either passed or failed. Along with the notification 
letter, every student who took the examination received a 

return post-card requesting certain information about his 

> 

training and his plans. Approximately 30 percent of these 
post cards were returned and the results of that suivey are 
reproduced- on- pages. 75 - 76. 

That office will continue to collect information about 
the students who were tested,' and pai ticularly about those 
students who will be entering and studying in the California 
State University and Colleges during the next several years. 

The results of this longitudinal study will be published 

> 

later . Among the questions to be answered by these follow- 
up studies are: 1) will students use the 6 units gained by 
the examination to abbreviate their college career , or rather 
to advance the level of their studies? 2) will students 
receiving credit for freshman English by examination avoid 
further English courses or rather be encouraged to- take pore 
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THE CALIFORNIA STATE UNIVERSITY AND COLLEGES 
English Equivalency Examinations - Spring 1973 
Descriptive Statistics - Post Card Response 



Total Response = 1,234 = 30.3%>of' Total Applicants 

Response From Those Passing = 1,216 = 98.5% of Total RfesjJonse 

Response From Those Failing. = 18 = 1.5% of Total Response 

Response From Those Passing = 1,216 = 89.3% of Total Passing 

Fall 1973 Educational Plans - Those Passing 

California State University and Colleges = 88.6%' 

California Community Colleges = 4.5% 

University -of California - 3.5% 

Other Colleges or Universities = 1.4% 

California Public High School = -2% 

No Response - This Item = 1.8% 

Institution Last Attended - Those Passing 

Public High School ' = 78.5% 

Private High School = 13.7% 

California Community Colleges - = .3.5% 

California State University and Colleges = 2.7% 

University of California = -2% 

Other Colleges or Universities = .3% 

No Response - This Item = 1.1% 

Taking "Enriched" English Course in High School 
Those Passing 

VES = 35.3% " 

NO = 63. 5% 

Mo Response - This Item 1.2^ 
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THE CALIFORNIA STATE UNIVERSITY AND COLLEGJES 
English Equivalency Examin'ations - Spring 1973 
Distribution of Fall 1973 Applications Those Passing 



Campus/segment Number Percent 

, Bakersfield ^ "19 1/6 

Chico . ,75 6.2 

Dominguez Hills , 7 * .6 

Fresno • 30 2*. 5 

Fullerton 113 . 9.3 

Hayward ' 4 6 3.8 

Humboldt * 102 8.4 

Long- Beach 71 5.8 

Los Angeles • - 18 .1.5 

Northridge^ 91 7.5 

Pomona 59 4.9 

Sacramento 41 3.4 . 

San Bernard^ino ^ 9 ' .7 

San Diego ' 120 9.9 

San Francisco ,80 6.6 

San Jose * 94 t . 7.7' 

San Luis Obispo 106 ~ 8. J 

Sonoma ' 4^ 3.^9 

Stanislaus '10 .8 

All Campuses 1,139 ' 91.7 

California Community Colleges 59 ^ 4.9 

University of California 98 8.1 

Other 34 2.8 

Tbtal Applications , 1,330 109.4 
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advanced English courses? 3) will students who receive credit 
for f^fesnman English by examination achieve as well in ad- 
vanced courses requiring v;riting as students with approx- 
imately equal ability who took the freshman English course? 
4) Do students with the particular skills and. abilities to 
pass this kind of test tend to go to or come from certain 
schools or certain programs? 5) to what degree is success 
on this sort of examination dependent upon race, economic 
level, or other ex^tr a- academic factors? 

V 

THE 1974 ENGLISH EQUIVALENCY EXAMINATION ^ 
As this report goes to press, word has been received 

that an examination sxmilar to the 1973 examination will be 

I 

supported by the Chancellor's Office, and the same adminis- 
trative staff will direct the CSUC examination for the 
English Council on May 11, 1974. 

: Administration of a large-scale testing program is an 
enormously complex, enormously time-consuming,' enormously 
expensive, yet enormously rewarding project* It is our hope 
that the program will in fact become a normal part of the 
procedures of highei* education, and that college bound 

students with the ability to profit from college-level 

i 

studies will find opportunities in high school to develop 
their skills in English as far as they are able. It is our 
hope and expectation that such students will be tested by 
English department faculty who are well aware of what is 
expected m their freshman courses, will be certified by 
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these faculty as having met these demands, will. receive 
credit for what they have learned, and will move into more 
advanced courses' with the enthusiasm and delight that 
successful students bring to a subject they love. Such a 
resul^t can benefit the teaching of English at both the 
secondary and college levels and can only be rewarding to 
^students and faculty alik^,. 
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VI 



CHARTS AND TABLES 

♦ 

A. Distribution of Composite Scores - Total Applicant 
Population 

B. Distribution of Composite Scores - Applicants who 
Passed" ^ 

C. Distribution of Composite Scores - Applicants who 
did not Pass 

n 

D. Distribution of Concerted ET^say Scores - Total 
Applicant Populati/n 

£• Distribution of Converted Essay Scores - Applicants 
who Parsed 

,F. Distribution of Converted Essay Scores ~ Applicants 
j who did not Pasi 

•G. Distribution of Objective Scores - Total Applicant 
Population ^ 

H. Distribution of Objective Scores - Applicants who . 
Passed '* 

I. Distribution- of Objective Scores ~ Applicants who 
did not Pass * ^ 

J. In-tercorrelations Among Part and Composite Scores ~ 
Total Applicant Population ' ^ 

K. Intercorrelations Among Grouped Essay and Composite^ 
Scores - Total Applicant' Population 

4 L. M-ale -Applicants by Age Category ^ * ^ 

« M. Female Applicants by Age' Category 
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THE CALIFORNIA- STATE UNIVERSITY .AND COLLEGES 
English Equivalency Examination - Spring 1973 



Distribution of Composite Scor.es ■ 
.Total Applicant Population 



Score 

51-59 
60-69 
■ 70-79 
80-89' 
90-99, 
100-109 
irO-119 
120-129 
130-139 
140-149 
150-152 



Frequency 
• 12 
96 ^ 
. 452 
908 
1064 
836 
445 
170 
52 
13 
2 



Total 



4050* 



Relative 
Percent 

\ 

0.3 

2.4 
11.2 
22.4 
. 26. 3 > 
20.6 
11.0 
'4.2 

1.3 

0.3 

0.1 
100.0 



Cumulative 
Percept 

,0.3 
2.7 
13.8 
36.2. 
62.5 
83.2 
94.1 
98.3 
99.6 
99.9 - 
100 . 0 



,Mean = 95.363 Median = 94.485 Mode = 91.000 

» 

\ Standard Deviation = 14.869 

*21 Applicants did not complete Essay Examination 
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THE CATiyORNIA STATE UNIVERSJTY AND COLLEGES 



4 

English 


Equivalency' 


Examination - 


stribution of Composite Scores for those Ap] 


Score 


Frequency 


Relative 
Percent 


1,00-104 


357* 


"26.2 , 


10>5-109 


331 


24.3 


110-114 


237 


• 17/4 


115-119 


201 


14.8 


120-124 


99 


7.3 


125-129 


70 


5.1 


130-134 


37 


2.7 


135-139 


15 


1.1 


140-144 


11 

• 


0.8 


145-149 ' ' 


2 


0.1 


150-152 


2 


• 0.1 


Total 


1-362 


ioo.o' 



Cumulative 
Percent 
* 

26.2 
50.5 
67.9 
82.7 
89.9 
'95.1 

i 

- 97.8 
98.9 
99.7 
99 .9 
lOO'.O 



Mean = 111.44? Median = 109.407 

Standard Deviation = 9.547 



Mode = 101.000 
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THE CALIFORNIA STATE UNIVERSITY AND COLLEGES 

'I 

f 

English Equivalency Examination - Spring 1973 

Distribution of Composite Scores of those Applicants who did not Pass 

> 



Score 

51-54 
55-59 ~ 
60-64 
65-69 
.70-74 
75-79 
80-84 
,85-89 
90-94 
95-99 
100-104- 
105-109 
110-114 
115-119 
120-121 



\ 



Frequency 

4 

8 

21 ^ ' 
"75 
166 
286 
402 
506 
' 557 
507 
116 
32 
■ 5 
2 
1 



Relative 
Percent 

0.1 

0.3 

0.8 

2.8 
' ,6.2 
,10.6 
. '-15.0 
18.8 
2 0.7 
18.9 

4.3 

1.2 

0 ."2 

0.1 

0.0 



,-' Cumulative 
Percent 



0.1 
. 0.4 
1.2 
4.0 
10.2 
20.8 
35; 8 
54.6^ 
75.3 
94.2 
98.5 
99.7 
99.9 
100.0 
100.0 



Total 



2688* 



100.. 0 



Mean = 87.222 Median = 88.202 

Standard Deviation = 9.472 



Mode = 91.000 



*21 Applicants Did Not Complete Essay Examination 
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■ THE CALIFORNIA STATE UOTVERSITY AND COLLEGES 
English Equivalency Examination - Spring 1973 
\ 

' • - * 

Distribution of Converted Essay Scoi;es 
Total Application Population 



' Relative Cumulative 

Score ■ Frequency • Percent Perc^ent 

22-26 11^ . 0.3 0.3 

27-31 . 67 • i.7 i:9 

32-36 273 6.7 8.7 

37-41 630, 15.6 24.2 

42-46 . 903 22.3 46.5 

V 

47-51 . 988 24.4 70.9 

52-56 654 -16.1 87*.l 

57-61 • 218 5.4 92.4" 

62-66 217 ' V 5.4 97.8 

67-71 75 1.9 99.7 

72-76 _ 14 0.3 ' 100.0 
Tot^l 4050* 100.0 . 

Mean = 47.916 Median = 47.662 Mode ^ 48.000 
Standard Deviation = 8.402 

*21 Applicants Did Not Complete Essay Examination 
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THE CALIFORNIA STATE UNIVERSITY AND COLLEGES 
EiTflish Equivalency Examination - Spring 19 73 

Distribution of Converted Essay Scores of those Applicants who Passed 



Re lat ive Cumulat ive 

Score Frequency Percent Percent 

48 171 ■ 12.6 12.6 

51 237 17.4" 30.0 

53 218 o' 16.0 - ■ 46.0 

56 26^ 19.2 65.2 

59 , 188 • 13.8 79.0 

62 113 8.3 ' 87.3 ^ 

64 86- * 6.3 93.6 

67 38 2.8 ♦ 96.4 

69 ^ 26_ • 1.9 9 8.3 

71 9 0.7 99. o' 

73 8 &.6 99.6 

76 6_ 0.4 100.0 

Total 1362 100.0 

Mean = 55.910 Median = 55.135 Mode = 56.000 

Standard Deviation = 5.800 
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THE CALIFORNIA STATE UNIVERSITY AND COLLEGES 
English Equivalency Examination - .Spring 1973 

Distribution Q.f Converted Essay Scores of those Applicants who did 



Score 


Frequency 




-Relative 
Percent 


Cumulative 
Percent 


22-26 


J 11 

< 




0.4 


0.4 


27-31 


67 




2.5 


2.9 


32-36 


^ 213 




10.2 


13.1 


37-41' 


630 




23.4 


36.5 


42-46 


903 




* 33.6 


70 .1 


47-51 


580 




21.6 


91.7 


52-56 


174 - 




6.5* 


98.1 


57-61 


30 




1.1 


99.3 


62-66 


18 




0.7 


99 .9 


67-69 


2 




0.1 


100.0 




Total 2688* 


f 


ioo.o 


« 




Mean = 43.868 Median 


= 44.130 


Mode = 46,000 




Standard Deviation = 6. 


369 


* 






*21 Applicants Did Not 


Complete Essay 


Examination 


csuc - 


Institutional Research 


♦ 


■ ■ > 
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THE CALIFORNIA STATE UNIVERSITY AND COLLEGES 
English Equivalency Examination - . Spring 19 73 



Distribution of Objective Scores 
Total Applicant Population 



Score 

22-26 

27r31 

32-36 

37-41 

42-46 

47-51 

52-56- 

57-61 ' 

62-66 

67-71 

72-76 

Total 



Frequency 
11 
80 

290 

665 

843 

883 

633 

364 

209 • 
• .75 

18 . 
4071 



Relative 
Percent 

0.3 

2.0 

7.1 

16.3 . 

20.7 

. 21.7 

15.5 

8.9 

5.1 

,^ "l.9 . 

0.4 

100.0 



Cumulative 
Percent 

0.3 

2.2 

9.4 , 

25.7 — 

46.4 

68.1 

83.6 

92.6 

97.7 

99.6 
100. o' . 



Mean = 47.4 48 Median = 47.140 

Standard Deviation = 8.85 4 



Mode = 43.000 
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THE CALIFORNIA STATE UNIVERSITY AND COLLEGES 
English Equivalency Examination - Spring 1973 



uis triDU tion 


or uDjeciixve 




of those Applicants who Passed 


Score 


Frequency 




Relative 


- ' 

Cumulative 
Peircent 


45-47 


113 




8 3 


8.3 


48-50 


218 






24.3 


51--'53 


283 




20 . 8 


45.1 


54-56 


186 




13.7 


■■ "58.7 


57-59 


194 




14.2 


73.0 


60-62 


164" 




'12.0 


85.0 


63-65 


91 




6.7 


91.7 


66-68 


/U 




• 5.1 


96.8 


69-71 


26 . 




1.9 


98.8 


72-74 


15 




1.1 


99.9 


75-76 


2 




0.1 ' ■ 


. lOd.O 


Total 


1362 




100.0 




Miean = 


55.577 


Median 


j 54.661 


Mode = 52.000 



andard Deviation = 6.318 
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- InsCi 
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THE CALIFORNIA STATE UNIVERSITY AND COLLEGES 
English Equivalency Examination - Spring 1973 

Distribution of Objective Scores of those Applicants who did not Pass 



Score 






Frequency 


Relative 
Percent 


Cumulative 
Percent 


22-26 






11 


0.4 


0.4 


27-31 






80 


3.0 


3.4 


32-36 






290 


10.7 


14.1. 


37-41 






665 


24.5 


38.6 ' 


42-46 






799 


29 . 5 


68.1 


47-51 






544 


20.1 


88.2 


52-56 






216 


• 8.0 


96.;^ 


'57-61 






74 


2.7 


98.9 


62-66 






25 


0.9 


99.8 


67-71 






4 


0.2 


100.0 


72-73 






1 


0.0 


100.. 0 




Total 




2709 


100.0 






Mean = 


43. 


361 Median 


= 42.951^ ^ 


Tlode = 43.000 



Standard Deviation = 6.914 
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THE CALIFORNIA STATE UNIVERSITY AND COLLEGES 

English Equivalency Examination - Spring. 1973 
Male Applicants by Age Category 



Age 




1" aiiea 






Passed 










Category 


N 


Rel.% 


Cum. % 


N 


Rel. % 


Cum.' % 


N 


Rel.% 


Cum . %■ 


16 and under 


1 


a.i 


0.1 


2 


0.5 


0-.5 


3 


0.2 


0.2 


17 


57 


4.4 


4.5 . 


20 


" 4.8. 


5.3 


77 


4.5 


4.7 


18 


1051 


81.5 - 


86.0 


322 


78.0 


83. 3 


1373 


80 .7 


85.4 


19 


152 


11.8 


97.8 


47 


11.4 


94.7 


199 


11.7 


97.1 


20 


10 


0.8 


98.6 




"'1.0 


95.7 


. 14 


0.8 


.9 7.9 


21 


3 


0.2 


98.8 


2 


0.5 


96.2 


5 


0.3 


98.2 


22 


3 


0.2 


99.0 


3 


0.7 


96.9 


6 


0.4 


9 8.6 


23 


2 

* 


0.2 

/ 


99.2 


1 


0.2 


97.1 


3 


0.2 


98.8 


' 24 


1 


0.1 


99.3 ' 


0' 


0.0 


97.1 


1 


0.1 


98.9 


25 - 29 


5 


0.4 


99.7 


6 


, 1.5 


98.6 


'll 


'0.6 


99.5 


30 34 


2 


0-2 


99.9 


2 


0.5 


99.1 


A 


0.2 


- 99.7 


35 and over 


2 


0.2 


100.1 


4 


1.0 


100*1 


6 


0.4 - 


100.1 


All Ages 


1289 


75.7 




413 


24,3 . 




1702* 


100.0 





*Age Data unobtainable from 15 (M or F) Applicants 
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THE CALIFORNIA STATE UNIVERSITY AND COLLEGES 
English Equivalency Examinaj:ion - Spring 1973 

Female Applicants by Ag^ Category 



Age Failed Passed " Total 



Category 

16 and under 


n' 
2 


Rel.% 
0.1 


Cum. % 
0.1 


N 

2 


Rel.% 
0.2 


Cum. % 
0.2 


N 

4 


Rel.% 
0.2 


Cum.% 
0.2 


17 


130 


9.2 


9.3 


74 ' 


7.8 


8.0 


204 


8.7 


8.9 


18 


1136 


80.6 


89 .9 


754 


79.8 


87.8 


1890 


80.3 


89.2 


19 


117 


8.2 


98.1 


69 


. 7.4 


95.2 


186 


7.9 


97.1 


20 


"7 ■ 


0.5 


98.6 


9 


1.0 


96.2 


16 


0.7 


97.8 


'21 


2 


' 0.1 


98.7 


6 


0.6 


96.8 


8 


0.3 


' 98.1" 


22 


1 


0.1 


98.8 


2 


0.2 


97.0 


3 


0.1 


98.2 




2 


0.1 


98.9 


1 


, 0.1 


. 97.1 




0.1 


.98.3 


24 


0 


0.0 


98.9 


6 


0.6 


97.7 


6 


0.3 


98.6 


25 - 29 ^ " 


"^4 


0.3 


99 .2 


4 • 


0.4 


'98.1 


8 


0.3 


98.9 


30- - 34 


1 


0.1 


99.3 


5 


0.5 


98.6 


6 


0.3 


99.2 


35 and over 


8 


0.6 


99.9 


12 


1.3 


99.9 


20 


0.8 


100.0 


All- Ages 


1419 


59.9 




, 944 


40.1 




2354* 


100.0 





*Age Data Unobtainable from 15 (M«F) Applicants 
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PREFACE 

* • 

By 

f> Edward M. White 

Chairman, Department of English 
California State College, San Bernardino 

^' 

In the fall of 1971, the California State Colleges began large-scale 
equivalency testing for entering freshmen at two colleges, using tests 
developed by the College-Level Examination Program (CLEP). (This program, 
sponsored by the College Entrance Examination Board [CEEB], is administered 
by the Educational Testing Service [ETS].) After the results had been 
publicized, serious professional evaluation of the .val idity , scoring, and 
administration of the tests began among the faculties; the State College 
English Council raised a number of objections to the English Composition 
General Examination in particular, as well as to various aspects of the 
program in general. The Chancellor's Office proved receptive to the English 
Council's objections, and to other questions raised by a series of statewide 
committees and subcontni ttees that have considered the Fall 1971 program. 
In late spring of 1972, the Chancellor's Office agreed to support a summer 
study to be undertaken by a committee of the English Council, to investigate 
equivalency testing in the area of English and' to recommend an appropriate 
program for use by the now renamed State University and Col leges. 

This report is the result of that study. Ifis not exhaustive, since 
such a task in this area would have demanded far more time and support than 
was available. It is an attempt to focus the major issues in such a way 
as to point to their solution, and it recommends a method of equivalency 
testing in English which is responsive to our discipline and practical to 
implement. 
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This report has passed through a serjes of drafts and presentations which 
have made it, in its present form, an expression of the best thinking of the 
English Council as a whole--perhaps even of the English profession as a 1 
whole. Since Spring 1972, when the Council directed me to prepare this' 
report, I have consulted widely with English department and freshman English 
chairmen throughout California, and have corresponded, sometimes at considerable 
length, with over two dozen specialists ^n the field elsewhere in the United 

4 

States and in England. I have kept ci^tations to a minimum throughout the 
report, ,^which is written for laymen as well as for -the professional, so I 
must thank here the many teachers, writers, and scholars whose published 
work and whose substantial and thoughtful letters to me havecontributed 
to our findings. I owe a particular debt to Professor Jess Ritter of 
Californio State University, San Francisco, who worked closely with me 
throughout the study, and to Dr. Albert Serling, Program Director for CLEP', 
who spent a week in San BernaYdino to give us tfie benefit of his wi^de 
experience. The English department chairmen and faculty who participated in 
the Southern California Advisory Meeting, August 3, 1972, and in the Northern 
California Advisory Meeting, September 14, 1972, will notice the many improve- 
ments made in the report as a result of their suggestions. I am also grate- 
'*' 

ful for the advice given me by William Schaefer, Executive Secretary of the 
^ Modern Language Association; Robert Hogan, Executive Secretary of the 
National Council of Teachers of English; and most particularly Michael 
Shugrue, Executive Secretary of the Association of Departments of English, 
who first helped me discover where to turn to dispel my previous condition 
of happy ignorance about the entire area of testing in English. 

ERIC 
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K EQUIVALENCY TESTING: The Central Issue 

Equivalency testing has become common practice in higher education, and 
^ has long been widely accepted, at \east in theory, in English departments. 
All but two of the forty-six four-year California colleges and universities 
responding to the 1971 Association of Departments of English Freshman English 
Survey, for instance, indicated that tnere was a way to exempt students from 
freshman English at their institutions. In addition, the Advanced Placement 
Program, also administered by ETS for CEEB, is widely accepted- as equi.valent 
to college work; a score of 3, 4, or 5 is accepted a'^s six semester units of 
college credit throughout the State University and College system. (See a 
memo entitled "Systemwide Policy on Advanced Placement and Credit" -sent 
by Vice Chancellor Langsdorf to all State College Presidents, June 16, 1971.) 

—But only recently has equivalency testing been open to very large numbers 
of students. Advanced Placement candidates, for instance, are relatively few 
in number, able and ambitious students, from a limited number of secondary 
schools; they enroll in specialized courses, and generally perform better 
than college and university students on their examinations and in their 
subsequent college work. Nonetheless, AP originally encountered considerable 
faculty resistance, and has become widely established and accepted only within 
the last decade. The CLEP program has greatly expanded opportunities for 
college credit by examination and hence has once agairv focused attention 
on the major theoretical issue raised by such credit. But since such large 
numbers are involved, the arguments have^become particularly heated. 

Those who argue for such testing assert that it benefits the individual. 
No one should be asked to repeat work in college that he has mastered; he 
should receive credit for what he 'knows and proceed to appropriate levels 
of learning. 
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'Those who argue against such testing also assert that the needs of the 
individual are primary. To substitute mechanical tests of competency for the 
individual search for excellence is in fact to cheat the student of possi- 
bilities for individual growth. 

These arguments, which can be and have been developed at great length, 
and which lead to rhetorical heights of passion, ])oint to the practical weak- 
nesses in both positions. Certainly college cgurses ought not to be rote 
repetition of what i-s already known, and certainly equivalency testing ought 
to lead to more advanced learning. When faculty argue against equivalency 
testing without much knowledge of available tests, or when testing people 
proclaim the uselessness of college course work without knowledge of the 
innovations and expansion of freshman studies, the conflict becomes severe. 
_(In an article on CLEP, The College Board News , May 1972,' claims the five 
general examinations afford freshmen "the opportunity to eliminate one 
entire year of study and exp.ense," which is a strange and sad way to speak 
of what is available in higher education.) There is plenty of blame to go 
around for a quarrel which is essentially foolish, and for which students 
and higher education in general must suffer. 

As in so many heated theoretical arguments, both sides are right, since 
they are talking about different things. Some of the tests that have been 
used are in fact poor and invalid; no one sensible defends them. Some 
college courses have apparently not been worth the taking; no one really 
defends them. But we need not and should not take extreme positions. 
No one could argue against a program of equivalency testing that satisfies 
these two conditions: 



in... 

1 ) the tests must be in fact college level ones, valid for their 
stated purposes, and properly normed--in short the tests mst 
gain academic respectability similar to that won by the Advanced 
Placement program , and 

2) the purpose of the tests must be so clearly seen that no one 

1 \ t 

can take them as a way to cheat students of their education 

by huddling them through credits to save cash; the tests need 

to be administered so that they in fact help students develop 

their fullest individual capacities , • o 

"Everyone stands to benefit from equivalency testing responsibly 
done. 

2. EQUIVALENCY TESTING IN FRESHMAN ENGLISH 

The issues discussed in Section One are more or less applicable to all 
fields of study, but they are most pronounced in the area of freshman English. 

It is no wonder that equivalency testing in freshman English is a 
long-standing problem. The course itself is a lohg-standing problem, 
nationwide. It is the most widely required college course (in 1970, 93.2 percent 
of all four-year college? and universities required at least one term of English), 
and a million or more students enroll in freshman English each year in this 
country. Yet there is relatively little agreement nationwide about what 
should be in such a course; while the most generally accepted intention is 
to improve students abil i ty to write, English teachers now use a large number 
of d-ifferent approaches, none of which is demonstrably certain of success. 
Since the course is itself in such an unsettled state, it is no wonder that 
so many of the testing programs are confused in purpose and in content. 

IDC 
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The sharpest problem for freshman Engl ish -courses fs one that relates 
directly, -to- the issue that divides us. about equivalency testing: is the 
objective of the course some kind of minimum competence, what Albert Kitzhaber 
called "immediate therapy for students whose -academic future is clouded by 
their inability to manage the wri'tten form' of English"? flr is the primary 
purpose "to focus the student; s attention on fundamental principles of 
-clear thinking and effective expression of that thinking" (Albert Kitzhaber, 
Themes . Theories , and Therapy . 1963,, pp. 2, 3.). The view of English as 
"therapy," as fulfilling its function by .imparting correct spel-ling and other 
conventional forms of expression, is widely 'held outside of the profession 
and even by 48.9 percent of the English departments in the United States 
(acco-ding to Thomas Wilcox, reporting on The National Survey of Undergraduate 
Programs in English, in College English , 6 [March 1972], 688). This is the view 
o,f freshm->n English assumed by most placement tests, with their heavy stress 
on errox-hiiOting and supposedly correct expression. But over half the pro- 
fession nationwide and all the English departments in the California State 
University and College system reject this vision of freshman English, in 
favor of Kitzhaber 's second view. Correct knowledge of formal English, 
valuable as it is for many purposes, is not all that is taught in our classes; 
hence such knowledge is not alone sufficient for equivalency. Our freshman 
English courses are more concerned with developing an awareness of the various 
levels of usage, which are appropriate to various situations, than in abstract 
notions of correctness; and we are far more interested in helping students 
develop and test ideas in writing than in maintaining the supposed purity 
of the tongue. 

Since freshman English has such varying objectives and definitions'", 
we should not 'expect any sir.^e national test, however reputable, to satisfy 

IDV 
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k 

the profession as a whole. We, however, need to insist that tests designed 
to examine minimal competence in mechanics, even when they are sound, do 
not do more than touch the periphery pf our courses. And we need to define 
as clearly as possible the objectives of our courses ^so that better testing 
programs can emerge. For reasons discussed in Section Five below, the 
nineteen institutions in our system have been»ab1e to come up with a far 
greater sense of agreement about objectives than has been possible nationwide, 

3. STRENGTHS AND WEAKNESSES OF OBJECTIVE TESIING " 

> 

The'whole issue -of objective testing is so complex,, and so much * 
research has been done on it, that to summarize the research risks super- 
ficial ty and error. Mos"t of the research' on this question has been done 
by the Educational Testing Service, which has been giving English tests to* 
large numbers of students since it was established in 1^8. Several general 
conclusion: emerge from the various reports produced &y the highly capable 

scholars ETS has employed in this area*: 

* fi 

1) Only those who know little about testing have unlimited faith 
in test ^scores; the specialists are well ^aware of the limita- 
tions and fall ibil ity.of any kind, of test, and 

2) the best test in composition will combine the most reliable^ 
elements of both objective and essay testing. 

All evidence shows tliajt both kinds of tests have important strengths 
and serious weaknesses; it is important to state here th^t there is no 
necessary conflict between essay and objective tests. We would, in fact, 
argue strongly against any equivalency testing. in freshman Bhglish that 
did not include both. 
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Here are five conclusions that we support in the area of objective 
testing in freshman EngTish: 

A. Most of the objective tests ava-ilable are poorj some scandalously 
so. We should not succumb to the feeling that peop-le who make up 
tests must know what is going on in the field of English; many of 
» them don't. Anyone wi,th ^knowledqe of mbdern linguistics or dialectology 

for instance, would fir)d some of the routine .questions about 
"correctness" or the locating" of supposed error's quite absurd. 
As one reads through test after test, he becomes convinced that 
the principal skill tested, repea-tedly, is the ability to take 
•tests, that is, the ability to discern the poipt of view of the 
test maker, and henc^"to guess" shrewdly the "right" answer. No 
wonder the results on such tests correlate .nicely with, success in 
' school, which is, after all, normally based on the same skill. 
In* short, the well-known deficiencies of multiple-choice 
testing still weaken most such tests. Here, for one example, is, 
• a question from one of the newest and mos-t popular tests' in English 
composition ^(sTightly changed to avoid copyright difficulties); 
■ it illustnates the typical bad' question still being written: 

English speaking ijiusicians use professionally large- 
\ numbers of words from which. one' of the following . ^ ^ 
I languages? 

a. German 

b. French - - ' ^ • 

♦ 

c. Spanish 

d. 'latin 

e. Italian 

The test makers are obviously, looking in this question for a 7 
scrap of information about the ways in, whicH'Eng.l ish .uses foreign " 
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words, in th.is case the Italian vocabulary for some aspects of 
musital notation. Some students may in fact pick up such 
information in a composition course, though it seems unlikely; 
but' the student mo'st able to fiTl in the proper square is likely 
to be the one whose parents wanted to and could afford to give 
him music lessons as a child. Those not so privileged (including., 
no doubt, some fine musicj.ans) are not likely to know the answer, 
regardless of their, writing ability. And someone^who knew too 
much—say a specialist in medieval music--might even give the 
"wrong" answer, Latin. ^ 

- At the same time,\a few objective tests ar* noticeably better 
than the rest, and we ought to guard against uninformed judgements 
about all objective testing. Sometimes committees responsible 
for developing. attest are wholly informed and up-to-date--some- 
"times, indeed, they are leaders in^ the field— and the test itself 
is sometimes reviewed with such elaborate <:are.that the routine 
problems of objective testirig are largely or .wholly removed. 
Writing abil'ity^'s'a hig|ilj< complex combination of many skills; 
objective test? measure some ski*lls analogous to and involved in ' , 
writing, but ca^n not measure all such-^kill s .arid hence can never 
be wholly valid*.^ We ought to distrust any. objective tes,t that 

'claims Xo test wnting ability in Us entirety, and we "should , 
inquire suspiciously intb* the' val idi ty of such claims. •^On the 

/Other hand, there are skills which are closely connected with writing 

abiltty (for example;, size* and' accuracy of vocjabulary, or reading 

♦ 

comprehensibn) which Lan be measured objectively with a high degree of 
reli*^bi1i'tyr^.We can and should demanci that any objective test 
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we use examine particular skills with demonstrated validity, . 
that it be free from the obvious flaws of such tests, such as 
social bias and ambiguity, and that it not advertise itself as 
testing more than it in fact does test. 

Within some important limitations, objective testing can be a 
hnghly reliable and economical method of measuring achievement. 
Dr. Paul Diederich, Senior Research Associate at ETS, and one 
of the most experienced scholars in the country on English 
testing, writes that he usually expects', when measuring a single 
test against a reliable series ofWriting evaluations, -"a corre- 
lation of about .65 with a good reading test, .55 with an objective 
test of writing skills, and .45 with grades on an essay given 
by trained readers under close supervision." These are dis- 
couraging figures: a correlation below .30 approaches irrelevance; 
professional .designers of objective tests aim for .90 and are 
distinctly unhappy below .80. But we must recognize the fact, 
demonstrated repeatedly,* that one good objective test will 
correlate more highly with a student's writing ability (using 
a series of writing samples as a base) than will one good ess^y 
test. This is a convincing argument-that the equivalency test 
we approve should contain an objective section. 
Since objective tests do not test writing ability directly; but 
only a few skills that are par.t of or associated with it, no 
objective test should be used alone as a measure of writing 
ability. It Ms essential that an essay be part of any writing- 
equivalency test that seeks to measure college-level skill . 

J. % 

1 

, 1 1 i 
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Everyone, even the most avid defender of objective testing, 
knows that some students can do well, or at least passably, on 
objective tests in writing and yet write abominably. In addition, 
whenever impersonal testing occurs, there are bound to be occasional 
instances of cheating, imperaonation, and other outrages endemic to 
a test-oriented society. For these very practical reasons, essay 

CP 

tests are needed to increase the validity and security of the 
whole testing process. 

Every English teacher's experience that writing ability is closely' 
akin to reading ability is borne out by correlation studies- (Note 
that Paul Diederich, as cited above, expects a higher correlation with 
writing ability from an objective reading test than from an^.objective 
writing test. ETS reports tend to confirm his expectation.) This 
finding supports the common practice of spending much time in freshman 
English on careful analytic reading of all kinds of writing, including 
but not restricted to, imaginative literature. Capable writers are 
almost always , capable readers, and i,t Is reasonable to expect that 
careful training in reading will help the development of writing 
Ability. Since writing and reading are generally linked in the 
course work, and improved reading ability is a. normal objective 
of freshman English, a test designed to give college credit in ^ 
the course must include a substantial reading component. It appears 
possible to test reading ability with some accuracy by objective 
examination, and it appears possible to test general reading 
ability at least in part by the us.e of . a valid and reliable general 
literature examination. But we must be careful that any reading* 
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test we use is college-level and substantive. It is simpler to 
ask for the correct spelling of Shakespeare's name (though 
Shakespeare himself would not know) than to obtain and evaluate^ 
a response to King Lear*s changing relationship to his daughter 
Cordel ia. . ^ 

« 

4. STRENGT HS AND WEAKNESSES OF ESSAY TESTING 

T " — ; 

"We ought to have no, illusions about the reliability 0/ essay testing. 
To be sure, it is the criterion of writing ab'ility; it is the only way to 
see the real thing. Nonetheless, such tests have many important problems 
of which we need to be aware. 

Perhaps the most significant problem for the reliability of essay tests 
is the large difference in quality of the essays written by a single student. 
Yesterday's paper is noticeably worse, of better, than today's and, of today's 
papers, the one on topic A is far superior to that on topic F. An essay 
does not measure writing ability as an abstract quality, but a student's 
ability to write on a certain topic on a certain day under test conditions. 
It is certainly conceivable that the student whose failing paper you may 
have read last night could have handed in his paper with a -bored sigh of 
relief, aWd gone home to write his girlfriend in Ci/camonga a witty, intelligent, 
mechanically accurate^ analysis of the test he had suffered through and of the 
agonies of the professor who would have to evaluate it. 

The second most important problem is the diff icul ty in achieving 
reliable gracing of essay tests. Even under the most carefully controlled 
and supervised reading conditions", it is hard to find readers who agree 
consistently about the quality of given essays. 'And the studies analyzing 
results iindsr more usual circumstances, when students are :wri ting on different 
topics, and when v/e know the identity of the writers,' are really depressing. 
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But it is possible to establish testing and'grading conditions to bring 
the reliability of essay testing to a useful point. It is clear that, as the 
ETS publication The Measurement of Writing Ability (1966) states, "The com- 
bination of objective items (which measure accurately some skills involved 
in writing) with an essay (which measures directly, if somewhat less accurately, 
the writing itself) proved to be more valid than either type of item alone." 

(finally, it is educationally necessary to require a student to write 
during any test of writing. We need to validate objective testing by guarding 
against students who may. have' learned to perform well on tests, but who cannot 
write competently. Suppose we were to choose a simple, wellrconstructed 
spelling test as the equivalency examination (we won't, of course). The 
first -ime we used it, the results might well be acceptable; most (but 
certainly not all) good writers happen to be pretty good spellers. But the 
next time, those students who did not "waste" their sqhool years writing, 
but instead studied spelling, would greatly improve their scores. In time, 
the exam might well stimulate mindless cramming and devalue the writing 
act itself. This would be the effect whatever combination of skills a non- 
essay test might examine; unless we include an essay test in our examination , 
we run the danger of defining writing as not-writing, and this would be a 
position without validity or integrity . , 

5. • EQUIVALENC-Y TESTING IN:FRESHKAN ENGLISH IN THE CALIFORNIA ^ 
STATE UNIVErSITV AND COLLESE SV^TEH " ] ' 

Conditions are favorable for the development of a responsible and 

accessible equivalency test in freshman English within the California 

State University and College system. Not only is the administration of 

the system on record as urging such testing in general, but the English 

Council itself has endorsed it in principle. In addition, some of the 

' " 1.1 4 
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problems. we have listed in relation to freshman English courses and to testing 
in these courses are much more easily resolved within the system than on a 
nationwide basis. 

For example, the contributions of the English Council to communication 
among the college departments has led to some general agreement about the 
objectives of freshman English in our institutions. Again, for various reasons, 
the English departments of the State University and Colleges have tended to 
devote a substantial portion of their time and some of their best energies to 
the development of freshman English. Hence, the nationwide neglect and frag- 
mentation of such courses has not been a major matter here; indeed creative 
experimentation, innovation, and the development of new materials In such 
courses have marked our recent history. 

The relative ease of communication among the 19 Institutions, the general 
seriousness and spirit of innovation with which we approach the course, and 
the substantial size of our combined student bodies all argue for the possibility 
of a well -planned and appropriately financed examination that could. have nation- 
wide implications." Indeed, the Importance of what we are here undertaking 
has not escaped the notice of ETS and CEEB; they have .given strong assurances 
-that the two organizations will bring their resources, experience, and ^knowledge 
to help us accomplish aims so consistent with their ;pu61;ic position on credjt- 
by-examination. The College-Level Examination Program^- has run into some important 
opppsitfon from faculties, most particularly faculties^^n English and mathematics, 
numbers of whom have found the general examinations in these areas unacceptable. 
In response, ETS and CEEB have recommended various uses of subject examinations 
in these areas and are developing new examinations in both fields. Those 
responsibile for CLEP are determined to regain* the confidence of these faculties. 
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We stand to benefit from a strong working relationship with ETS, which has 
done most of the valuable research in testing in our field, since this 
accumulated expertise (though by no means infallible) is an invaluable 
resource. 

The testing program we recommend has four features to it, each of which 
is discussed below: A) A coherent statement of the aims and objectives of 
freshman English, B) A test, including- both objective and essay parts, which 
is demonstrably responsive to these aims, calls for an appropriate college 
level of proficiency, and is valid and reliable, C) Administration of the 
test reliably and professionally, and D) Professional and sensitive use of 
test results. Such a program is-liot only academically sound, but financially 
and technically practicaj^we propose it go into operation for the fall of 
1973, with initial testing to begin as early as spring 1973. 

A. Aims and objectives of freshman English 

Freshman English calls for development ot reading and writing 
ability--including the effective uses of reference and resource 
materials^-as well as the acquisition of knowledge about the 
English language. A student should demonstrate the college-level 
ability 

1. to recognize and use appropriate language (rather than merely 
to classify "errors"), 

2. to recognize and use the basic processes of clear thought ' 
and clear communication, and 

3. to read expository and imaginative Writing with understanding. 

B. The Test: Objective and Essay 

The test should contain both essay and objective parts. Six 
semester units of lower division credit, or its equivalent, * 
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should be given for successful completion of an examination 
of 3 hours, consisting of 90 minutes of objective testing and 
90 m'ihutes of a carefully designed essay test. 
Administration of the test 

1. Proposed new CLEP Freshman English Test: Fall, 1974 

We -have great hopes that the proposed new CLEP F.reshman 
English Subject Examination will be satisfactory f,or our 
purposes. We have confidence in the committee of? Examiners 
devising the test (Richard Braddock, University of .Iowa; 
Greg Cowan, Forest Park Community College, Missouri; Marianne 
D&vis, Benedict College, South Carolina; and Walker Gibson, ^ 
University of Massachusetts) and respect the committee's 
statements about what it is seeking to accomplish. In addi- ^ 
tion, we have examined six 45-minute pretests containing 
approximately 450 questions written by col lege"Engl ish 
teachers to the specifications of that^ committee. These 
pretests constitute an item pool from Which about 200 questions 
will be drawn to yield two editions of 90-minute CLEP multiple 
choice subject examinations. On the following page are the 
test specifications developed by the cormiittee of examiners. 
The questions on the pretests seem specifically designed to 
avoid the usual faults of short-answer testing, and seem 
generally to examine the kinds of ski lis, we have agreed 
are among our most important objectives. 

In addition the new CLEP freshman English test includes 
a 90-minute optional essay section which we can and should 
require. The committee preferred a required essay section 
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as part of the test itself, but CLEP's policy is to let the 

decision on requiring the essay rest with the institutional 

scoVe recipient. Everyone involved in creating the^ test agrees 

upon the value of the essay, however. Here is the po\icy of 

the CLEP program in relation to essay testing for the new CLEP 

Freshman English Test (exerpted, with permission, from an ETS 

memorandum dated July 14, 1972): 

"The CLEP Program can offer a most positive alternative 
in the special case of this new freshman English test. This 
will permit and promote the careful, rational use of the 
optional essay section without penalizing those candidates 
whose essays would be misused or ignored: 
"(1) If the committee makes its strongest 
possible recommendation urging recipient English 
departments to require the essay, the program will 
develop and distribute widely a special publication, 
aimed at college faculty members and departinents , that 
will highlight the commrttee*s recommendation. Colleges 
ancross the country are in the process of developing 
policies of credit by examination through CLEP. A 
strong recommendation by the committee that this test ^ 
is incomplete without a carefully prepared and graded 
essay should be, we think will be, welcomed by most 
recipients of scores. These schools can, should, and 
will in turn make it clear to individuals ^seeking 
credit that the essay is required by the recipient 
institution ." 

We expect to follow the development of this new test with 
keen interest, and are prepared to recommend its use if it . 
fulfills its promisee. We will seek to be included in the 
norming studies for objective portions of this test in the 
spring of 1973, and we will explore ways to conduct ^:oncomitant 
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norming of the optional essay section for stud|^nts in our 

system. We have been assured by the Director of the CLEP 

program that the program will make tests available to us for 

these purposes at no charge.,, and will assist us in our validity 

studies. Unfortunately, while CLEP designs .and provides for 

an optional essay, the receiving, institutions must themselves 

provide for the grading of the essay question. Therefore, 

funding from the California State University and College 

budget will be needed in the 19-72-73 fiscal year to establish 

an organization to read and evaluate essays for this test (or, indeed, 

for any other); this arrangement must be carefully and professionally 

set up, so* as to assure the reliability and validity of the entire 

program. We expect to be able to draw upon California faculty 

experienced in AP and other organized essay grading efforts to 

assure the professional caliber of this essential operation; 

ETS specialists in this area stand ready to assist us. 

However, because of the elaborate evaluation this new CLEP 
test will undergo, it wj.ll nOt be available for our use in 
September 1973. We thus need to choose an acceptable alterna- 
tive for the year ahead, even as we watch the development of 
what may well be a CLEP test we can accept without qualms. 

Analysis and Interpretation of Literature: Fall 1973 

We recommend the following as a responsible short-term ^ 

solution for the 1973-74 academic year only: 

A three-hour examination consisting of the 90-minute 
objective CLEP Subject Examinatipn, Analysis and 
Interpretation of Literature, and either its 90- 
minute essay section or one of our own devising. 

.1 

120 
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The disadvantage of this short-term solution is that the 
test does not deal with composition aside from literature, 
and that no norms /.have been developed specifically for our 
student population. 

The advantages of this proposal, however, are important: 

a. The test exists, and has been well received throughout 
the country and within our system^ It contains a highly 
reliable and valid objective test .{according to the 
elaborate studies conducted by ETS), which will serve 
the necessary measurement function of the objective 
portion of our test. • . . 

b. The Literature test, while not ideally suited- for all 
aspects of freshman English, is skewed in the direction 
of rigor rather than ease. It is a college level 
examination. 

c. Reading skill correlates closely with writing skill, 
and this carefully constructed reading test, along with' 
a 90-minute essay test, is more appropriate for our 
short-term use than any objective so-cal led .composition 

teSt. 

d. Two new, up-to-date, editions of this test will be 
available for^our use in 1973. These new editions 
will improve an already impressive test. 

e. CLEP has no objections to substituting an essay test ' 
of our own devising for^^the essays on literature now 
part of the examination* We can select appropriate 
essay questions for our purposes as the testing date 

X21 
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approaches, or we can accept those prepared by the CLEP' 

committ^ee (William Vesterman, Rutgers University; 0. 

Sutherland, University of Texas; Mary "kion Hove, S't.^ OTaf 

College) with the advice of the ETS test special ists. 
Essay Grading: Superv.ision and Expenses 

We resolve that the English Council will select a committee 
with confinuing responsibility for supervision* of the testing 
program. We; need furlfher reports on the development of the 
*n'ew CLEP Freshfnan English test, and since there is no national 
grading system for CLEP essays, we need to supervise the entire 
process of essay grading. ' . " 

We apropos e that the English Council , /funded through the 
Chancellor's Office, take responsibility for evaluating the 
student essays written for course equivalency in English,^ We 
can as a body ensure the integrity, consistency, ,anjd quality of 
essay grading far better than can any other office. Since^ 
essay grading is complex and expensive, it is bound to be' 
vulnerable; under our direction it will be less dssajlable,. 
less costly,, and more reliable than any but a national system' 
such as used by AP. , ^ * 

The cost of reliably grading large numbers of , essays is 
not prohibi'tive (about $6.00 per exam, based on tentative ' 
estimates of costs shown in Appendix I); when measured against^ 
the potential savings for students and the system, and .when 
placed against enhanced recruitment of able students, this 
expenditure in fact becomes a great bargain. 

.122 • . . 
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The cost oV developing the examinations we recommend ^ 
are being borne by CEEB; the cost bf taking the examination 
is borne by 'the candidate seeking credit; the costs of scoring, 
reporting, a'nd transcript service for the objective test^are 
provided by ETS;' the cost of scoring and ^sing the essay section 
of the test needs to be provided b/ the State of California. 
During the 1972-73 fiscal year, this cost should, we suggest, 
be paid by the fund for innovative programs. But after the 
1972-73 fiscal year, the faculty staff ing*- formula should pro- 
vide for the program, which ooviously calls for continuing 
attention from the English Council' and for maintai'ning a ^pdol 
of trained readers. We hope that costs of grading' can be ^ 
reduced,^ as we gain experience; it may be that the scores 
on the objective test will be so valid for our purposes that 
papers of those^on the upper and lower end of the scale will 
not need to be read. 

See Appendix I for a 'tentative budget, tabulating the 
anticipated expense of grading 5,000 90-minute essay 'tests. 
Passing Scores 

We accept the recommendation of the independent Council 
on College-Level Examinations for the acceptable passing 
score on the objective part of the tes*. The Council recommends 
credit, be granted for scoVes at or above the mean score for C 
students on the CLEP national norm. For the Analysis and 
Interpretation of Literature tests,. that is a score of 49 or 
roughly the- 56th percentile. (We may wish to use a California 
rather than a national mean score, when such local norming 



takes place.) ' The. ess'ay test will need to -be scaled by the 
chief reader and his .assistants. after theTcoring has been done,' 
and the two scores combined. ' " . ' ^ j 

The Use of Test Scores ' > 

/> ■ \ 
'.The use of test" results requires- careful at'te'ntion and 

planning, those who have-passed the test and]. received credit - 

for the college course work should be fully informed o"f the . . 

value of what they have achieved in 'academic* and devel'opmental ^ 

terms— not m'erely mechanical or financial langMagej;^ they should 

be urged to take more advanced work in Engl'ish in order to develop 

their capacities further.' Thus the placement value of this . , 

kind of testing should be exploited, even if course equi-va.lency . 

IS the major purpose. . ' " - . . 

< ^ 

- The individual colleges .and universities should also retain 
flexibility in the use of test -resul ts,, even if cre4it is^ 
granted systemwide. A'stude.nt who does not succeed in passing ^ 
the equivalency examination may wish to' apply for a challenge 
examination at his own institution; lie Ghould have the*'6pportuni ty 
to do so, if the institution Wishes to continue offering such 
tests on a local basis. ^ ^ ' - 

The student should have the option of how he will use crejlit ' 
gained by examination, the experience of AP students is illus- 
tr'a.tive in this regard;- these students, with their head start,, 
take more college units than do students without AP credit. 
.Certainly, careful and sensitive counseling, advisement, a,nd 
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guidance are e'ssential to this program, and not only for 
those likely to be successful in it. Those with little 
chance of success ought not to be encouraged to take tests 
covering college-level work they do not know- those succeeding 
at the tesTs should/be encouraged and guided in their self- 
motivated quest for learning.". Decisiom. however, must always 
rest with' the student, and each institution should seek to 
develop appropriate ways to help the student decide wisely. 
The Colleges and the Schools •- ^ * 

Since it is not to^be expected that most, or even many, high 

school graduates will in fact have accomplished college-level 
work in English, without special training', ho equivalency test 
program is complete without close liaison between the colleges 
granting credit and the schools. For a college-level equivalency 
program to succeed for more than a few individuals with unusual 
training or talents.- the high schools will need help and support 
.in providing formal col-lege-level opportunities for all students 
who may profit from such opportunities. Such an innovative ^ 
approach requires not only subject field communication between 
the colleges and 'the schools, ^but also a deliberate program of 
action on the part of 'the Chancellor's Officer and the State 

• Department of Education. We urge those agencies to initiate 
-and foster a .large-scale effort to assist the schools in 
establishing appropriate curricular offerings, so that the 

' equivalency program We recommend can in fact be open to all 

- pb'tentially qualified students. 
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6. HOW EQUIVALENT IS EQUIVALENCY ? 

Even as we- endorse equivalency examinations and proceed in all good 
faith to administer them, we need to reassert the value of our freshman English 
^ programs. After all, only a small percentage of our entering freshmen are 
likely to have learned college level skills in pur field, and even some of 
those receiving credit may well seek to take freshman English in order to 
receive the less measurable benefits of the course. 

' Freshman English, as well as rrtany other college courses, offers various 
experiences that have little to do- with measurable skills, and yet that 
can be of great educational value to students. For example, as Thomas Wilcox 
puts it, "The English class may offer the freshman his only opportunfty to 
participate in^the free exchangee of ideas and confer with, professional 
intellectual. This may be the best reason for limiting the size of freshman 
English classes and, indeed, the chief justif ication*of freshman English 
itself." At a time when humanizing higher education has become much more - 
than a slogan, we should not overlook the'liumanizing effect of a good fresh- 
man English course. "Students often testify, as they look back, that their - 

freshman English course first brought their minds to life Because 

freshman English classes are still relatively small in most institutions, 
the instructor is often able to provide individual help for the student; 
he often becomes a counselor as well as a teacher, just because he is less 
remote than the lecturer in the large introductory courses." (Robert Gorrell,^ 
"Freshman Composition," The College Teaching of English , ed. John Gerber ' . 
[New York,] 1965, p. 92) ^ 

If equivalency becomes one more mechanical device to turn education into^ 
processing, we will have done our students and our society a significant dis- 
service, even if we have saved them some cash. 
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. If equivalency becomes a siijiple matter of certifying minimal competency, 
without a concomitant push for achievement of individual excellence, we will 
have denied' our mission. 

We need to hold fast to our purpose as educators of individual students, 
even as we must get involved in the machinery of testing for units. The surest 
way for us to keep equivalency.. testing to its stated purpose of fostering 
and individualizing education in our field is for us to supervise directly 
a responsible professional program such as the one we here propose. Our 
aim, after. all, is to help students educate themselves; we should expect that 
students will continue to come to us for the best we have to offer, and we 
can certify their achievements in various ways. Equivalency test scores 
may well be equivalent to- our course grades, but the full and rich experience 
of language and literature, however measured, has no equivalency. 
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EXPENSE OF ESSAY READING 

Following are estimates of the expense.of reading with reasonable 
reliability 5,000 ,90-nii nut e examinations, each composed of three separate" 

essay questions. These estimates assume the followi^ng: 

a. Three independent readings wijl be given each paper (one reading 
for each question). ^ 

b. Five minutes of reading time will be required to score each 
essay- or a total of 15 minutes for each test, 

c. Six tables of eight readers and one table leader each will be 
required for the reading; two tables for each question. 

d. Each reader will receive an honoranium of $300 for days of 
work; each table leader will receive $350 for 5 days work. 

e. An experienced chief reader will organize and direct the reading; 
$1,000 should cover his honorarium, travel, and expenses. 



Honoraria 

(48 readers @ $300 and 

6 table leaders @ $350) , ^ $16,500 

Per Diem^ ^ " . 

(Housing and meals 9 $22 per day) 5,412 
Transportation 

(Average $50) ..2,700 
Chief Reader 

' (Honorarium and expenses) 1,000 
Clerical and Data Processing 

(Combining the 3 separate scores; ^ 
combining the total essay and 
objective test scores; weighting 
scores appropriately, etc.) 5,000 

. TOTAL $30,612 
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OBJECTIVE TESTS IN FRESHMAN ENGLISH 

* * * 

The following objective tests were made available by publ ishers'for 
examination by the Writers of this report. The College Proficiency Examination 
Program (CPEP) Examinations created and used by the University of the State 
of New York, were not made available;- there are, no doubt, other tests in use, 
or in potentia , that we have not seen. We did, however, attempt to examine 
every widely, available test designed for freshman English. 

.• American Guidance Service, Circle Pines, Minnesota 

Essentials of English Tes,.t -(forms A and B), by Dora V. Smith 
and Constance M. McCul lough, rev. 1961 by- Carolyn Greene 

Bobbs-Merrill . New York , 
Analytic Survey Test in English FundamenLals (form 4), by 
.J. Helen Campbell and Walter Scribner Guiler 

Bureau of Educational Measurements, Emporia, Kansas 

'Barrett-Ryan English Test (forms I, II, -III, VI, 1948,' 1954) 
Barrett-Ryan- Schrammel English Test (forms EM, DM) 
Hoskins-Sanders Literature Test (forms A, B) 

Walton- Sanders English Test (Test I, form B; Test II, forms A, B) . 

Educational Testing Service, -Princeton, New Jersey 

CLEP General Examination, English Composition 
« 

CLEP Subject Examinations 
English Composition 
American Literature 
English Literature 

Analysis and' Interpretation of Literature (six forms) 
Freshman English (six pretests) 

Undergraduate Program (UP) ' 
Literature Test 

European and American Literature Test (modular compVement 
to the Literature Test)'' 

Cooperative English Tests (forms lA and PM) 

Harcourt,. Brace and World; New York 

Missouri College Engl ish^ Test\ by Robert Callis and Willoughby 
Johnson (form-B) 



124. 



Houghton, Mifflin, Co.; Boston, Massachusetts 

The New Purdue Placement Test in English (forms D and E), by 
G. S. Wykoff, J. H. McKee, and H. H. Remmers 

McGraw-Hill, Monterey, California 

- Test of English Usage, (form A), by Henry D. Rinsland, Raymond W. 
Pence, Betty Beck and Roland Beck 

Educational Skills Tests, College Edition: tnglish (form A) 

Psychometric Affiliates, Brookport, Illinois • 

College English Test (forms A and B), by A. C. Jordan 
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LIST OF CORRESPONDENTS 

The authors of this report are particularly grateful to the following 
specialists in the field of testing in English, for their detailed and 
valuable letters. If the report is valuable to the profession as a 
whole, it will be in large part dlje to the partic^'pation of the pro- 
fession as a whole. 

G. C. Allen, Deputy Director, School of Education, University of 
Sussex, Brighton, Sussex, England. 

Richard Braddock, Coordinator, Rhetoric Program, University of 
Iowa and Editor of Research in the Teaching of English , Iowa 
, City, Iowa. 

J. N: Brittbn, Goldsmith Professor of Educationi University of 
London, England. 

Rex Brown, Assistant to the Director, Research and Analysis, National 
Assessment of Educational Progress, Denver, Colorado. 

Jonathan Bryan, Associate Professor of English, Northern Virginia' 
Community Co-llege, Annandale, Virginia. ^ 

Michael Cartwright, Coordinutor of Freshman English, California 
' State Collegev Bakersfield, and Director of the Association 
of , Departments of English Survey of Freshman English 
* * Programs. 

Paul B. Diederich, Senior Research Associate, Educational Testing 
Service, Princeton, New Jersey. 

Gerhard Friedrich, State University Dean. Academic Program and 
Resource Planning, .;,pff ice of the Chancellor,. The California 
State University and Colleges. 

Walker Gibson, Professor of Engl Ish, ^University of Massachusetts, 
Amherst, Massachusetts. 

William F. Irmscher, Director of Freshman English, University of 
Washington, Seattle » Washington. 

Arnold Lazarus, Professor of English, Purdue University, Lafayette, 
. Indiana.^ 

William J. Leary, Professor of English, California State University, 
Los Angeles, California 

Stoddard Malarkey, Director of Composition, University of Oregon, 
Eugene, Oregon. 

F. T. Naylor, Director for Advanced Level Examinations,, Schools , 
Council, London, ^England. 

Alan Purves,' Professor of Engl ish,* University of Illinois, Urbana- 
Champaign, Illinois* 

J. Stephen Sherwin, Chairman, Freshman Program, SUNY College at 
'Buffalo, New York. 

Blanche J. Skruriick," Acting Director, Basic Writing Program, The 
City College of the City University of New York. 

• , ' ? 

9r W. 0. S. Sutherland, Professor of English, University of Texas, Austin, 
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' TO: Dr. Robert Bess, Director DATE: Djecember 4, 1973 

Academic Projects 
•^Office of the Chancellor 

FROM: R. W. Lid' and Edward M. White 

* English Equivalency Test ing^ Program 

( Attached Is t?he full budget foe the English Equivalency Testing Program, in- 

cluding funds already allocated and funds requested. Thus far we have been granted 
$15,741.00 start-up costs. We are requesting an additional $60,313.00 for^ a total 
of $76,054.00 

For convenience the budget has been itemized under three separate parts: 

(1) Start-up costs, allocated and requested' 

(2) Reading costs requested 

(3) Follow-up costs requested. 

The third category runs beyond the, current fiscal year. 

As to the request for additional funds under item (l)*above, the original 
budget wa^ for the Fall semccter only. Within that budget ye chose to economiz,e 
by stretching faculty assigned time through a second quarter. Hence the requested 
additional reassigned tifie is for one quarter-course' only , together with an ad- 
justment figure for the semester/quarter differential. Beyond the above, there are 
some added clerical and operating costs which occur as we approach the reading 
period. A) so, the State Printer* s estimate for the printing of the brochure is 
running higher than anticipated, probably because of rising paper costs. 

The Reading Conference costs are all based on 6,000 candidates taking, the. test. 
Should there be a substantially larger number, we would have to request additional 
funds — unless polity were set in advance that we close registration at some point. 
Conference costs are figured at a substantially lower figure than last year based 
on workload and production figures for Summer 1973. In effect, we will be reading 
half again as many test booklets (6,000 instead of 4,000) for only a marginal in- 
crease in costs (3-4 •/„). If we were to »go over the estimated 6,000 candidates, the 
additional cost would be five dollars per test booklet to 7,500. It is of course 
assumed that if we come in under the 6,000, we will hire proportionally less readers. 
Finally, It should be pointed out that transportation and per diem have probably been 
figured high; we have been cautious because the gas shortage may mean that fewer of 
our readers will choose to commute this year. 

As to Follow-up costs, they represent DrT White's seeing the final processing 
and mailing of results to their conclusion. They also include the writing of a 
detailed final report and the making of* a statistical study. 

Two final ppints: (l)^LaSt yea): Dr. Smart offered the Deans of Admission pcst- 
• age costs for mailing the brochures.'^ Long. Beach billed* us $245. OOi Had others done 
the same, ve would have quickly igone over our budget. We would like a policy for 
this year that excludes such' billing; if not, then we would like additional funds 
granted for each such charge mado agdirir^f. us.. (2) The only new item in this budget 
over last year's is computer costs. We have been unable to ascertain them precisely 
as of this dafe and tho figure f^ivtin is' an estimate. 

The budget has been itemized in some detail. Should you have any questions or 
need addicional data, please feel free to ask Dr. Lid for such information. 
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Englis4i Equivalency Testing Program 



Budget Summary 



Start-up costs $21,515.X)0 

Reading costs 49,348.00 

Follow-up costs.... 5,191.00 

$76,054.00 
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Start-Up Costs 



Original 'Grant • Personal Services 



Faculty Man Years: .50 $ 6,252 

Temporary Help 3,415 

Staff Benefits (42r.) ' 1,018 

Salary Savings (194 ) 

Total Personal Services $ 10,491 

Operating Expenses 

Brochures . $ 1»700 

Mailing ' \ 1,500 

Supplies 500 

» Facility Reservations 100 

Word-Processing - 200 

^ Travel 1,250 

Total Operating Expenses* . $ 5,250- 

iTotal Budget $ 15^,741 



Additional Costs - Personal Services 



^Faculty Man Years $ 2,432 

Temporary Help ' 1,000 

Staff Benefits 292 

Total Personal Services $ 3,724 
Operating'. Expenses 

Brochures , $ ^^0 

Mailing ' 200 

Dittoing 2ac 

Xeroxing ^ JOO 

1 1.000 

Travel »- 

Total Operating Expenses $ 2,050 

Total Additional Request $ 5,774 



iThis is 'the original budget (Ref. No. A73.217) of $15,004 as adjusted to include 
7.5 salary increases requested/received by Northridge. 

2This sum is needed to cover disparity between quarter ."^^.^^^^^^""^^"wute 
systems ($1,042) and one additional course of for Dr. White ^^^g^ 
is to have 4/9 ths reassigned time (4 courses); Dr. Lid, .25 course.; tor ai. . 



Rcadi'hg Costs 
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Summary 



Reading Costs 



Readers and Professional Team • " $ 27,300 

Aides ' , . ^ ^ '2,301 

Per diem/RQaders and Professional *Team 9,850 

Tiransporeation/Readers & Professional Team 5,'850 

Lunches /Aides , . ^ 247 

Registration Fee Kellogg-Wesr 1,500 

Computer costs • 2,000 

Xeroxing ^ 250 

Telephone * * ^ 50 

Total . * $ 49,348 



Costs Itemized by Category 

r * 

I^eaders and Professional Team 

60 readers (? $300 (4^ days) • $ 18,000 

Itf table leaders @ $400 (5 days) 4,000^ 

2 question leaders (? $500 (5 days) 1,000 
1 chief reader (12 days) 1,200 
1 associate chief reader (17 days) 1,700 
1 secretary (6 days)* 300 

*1 statistician (6 days) 600 

• _2 chief aides (? $250 (5 days) 500 

* $ 27,300 

Aides * • " . ' 

10 (a $2.00 pe-r hr., 9 hr. day, 5 days • $ 900 

5 © $2.50 per hr., 9 hr. day, ^ days , 675 

3 (3 $3.00 per hr., 9 hr. day, 6 days 486 
(3 $2.00 por hr., 6 hr. day., 5 days 240 

22 $ 2,301 



*For 1974 reading ETS will likely provide statistician and travel 
cpsts; we will provide per diem. Hence this particular sum will 
most likely not be expended. 
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P^r diem/Readers and Professionab Team , 

74 (? $25 per day for 5^ days $ 9,250 
4 (? $25 per day for 6 days 600 

78. ' , ^ $ 9,850 



Transportation/Readers and Professional Team 

78 @ $75 average $ 5,850 



Lunches /Aides 



14 for 5 days (? $2,50 per day 
8 for 6 days (? $2,50 per day 

22 



$ 



175 
247 



Registration Fee - Kellogg-West . 

78 Readers and Professidhal Team & 
22 Aides (100 x $15.00) ■ ' 



$ 1,500 



Computer 



Xeroxing 



Telephone 



$ 2,000 
$ 250 

$ " 50 



=Rir 



130 



. J 



131-. 



Follpw-up Costs - -Summer 1974 



•Director's Salary (2 months) $ 3,742 

Terapdrary Help • 1,000 

Bfineflts 449 



1-3 V 



$ 5,191 
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The C-ali'fornia State University 
English Equiv^ency Test 

' . " STATISTICAL PROCEDURES 

.May, 1973 . . William Coweli 



I ft Thci Test and Scores : * , . , 

The California State University English Equivalency Test con-' 
sists bf tvo ninety minute parts; an objective part and .an 
esray paVt-. The object'ive part is the Subject Examination 
"Analysis?- and Interpretation of Literature" pf. the College- 
Level Examination Program (CLEP). The essay pj^art^has »been pre- 
pared by the Engli sh . pr-of es sors of the Call;forna.a' Stute Univer- 
sities and Colleges. • . ... 

Educational Testing Service will score the objective pa*rt and 
report scores on a scale of 20 to 80 . The essay paxt consists 
of two questions. Each of the two essays will be read by two 
readers. Readers will 'assign ^grads bn a scale of 1 to 6 (zero 
•will denote that t-he student did hot attempt an answer). The 
^ four essay grades will be added to obtain the essay total score. 
Essay totals can range from 0 to 21;. Essay totals of 0-3 are 
poss^^ible only if the studjent does not attempt one or both of 
the questions. ^ • " 

*A Cpmposite score for the v;hole examination (objective agd' • 
essay) will.be determined by transforming the essay total score 
to the 'same scale as the object^ive scores and ad'ding the objec- 
" tive and the trfiinsformed essay scores. The te'chni que> f or trans- 
forming the essay scores iss^escribed below (Sec-tion III). 

1 1 . Candidate Groups and No rms Samples : 



f 1 ) Total Group : 



The candidates (potential Calil^ornia State University and 
College freshman)- who register for and take the English 
Equivalency Test. 

2) PJquating Sample : . 

A sample of the Total Group. Since it is desirable 'to begin 
the work o.f combining scores befor.e ^the essay readTng is 
finished, the transf ormation^ of the .e.ssiiy scores to* the CLEPt 
will be based on a sample of the total group. This sample 

0 ' " . 
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vill consist, of the first 500 papers fr^om the total group 
to have^ the reading .comple t ed . . ^*Sinc e .the papers of the, 
total group will be "scrambled -be'f ore tlie reading begins,^ 
the sample should T>e representive of the total group. The 
.representativeness can bd checked by comparing the o.bjective 
scores of thei-sample ^nd. total group-.^ If the mean or stand- 
ard deviatiori of the scores for the sample differ signifi- 
cantly from those of the total ^group, the sample should be 
increased until it is represent'^at iye . This sample will be 
referred " to as "The Equating Sample" . , 



3 ) CSUC- Norms Sample 



" The samj>le of CSUC students '^who take the essay part of the 
•examination as part of their ilnglish course. This group 
will be used in determining minimum passing scores. 

U) CLEF Norms Sample : ' ' ^ 

' The nation-wide sample of college students who participated 
in the 196k norming administration of the CLEP Analysis 
and Interpretation of Liter.ature examination. 

5)^ Texas Norms Samplg : J 

; The sample of students who participated in the 1970 validity 
stuay of the CLEP examination at the University of Texas. 

III. Equa t ing pf .Ob.jective and Essay Scores : 

Equ^^t.ing is a method of transforming the essay stores to the 
CLEP scale to permit addition of objective and essay scores in 
a meaningful way. Equating 'may also be of value in d^etermin- 
ing minimum p.assing points. , ^ * . ^ 

Since both objective and essay scores vill be available for the 
Equating Sample (see IT abovq) , it is possible to establish a 
_/table showing the CLEP score (on a scale of 20 to 80 ) equivalent 
"to each essay. tot^il score. The process of obtaining this table 
is called "score equating". Severa^l acceptable methods are. 
available . The equi-percent ile (curvilinear) method seems to 
t be most appropriate for establishing comparable objective and 



Angoff, W. H. ; Scales, Norms, and Equivalent Scpres- in R. L. 
Thorndike (Ed.) "Educational Measurement". (2nd ed.) Washington, 
D.vC: American Council on Education, 1971, 563-590. 
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essay * scores . The scaled-score corresponding to each essay 
total score will be that score in the distrib.ution of objective 
(CLEP) scores such that the^pe rcentage -of the group scoring 
below that CLEP score ife 'equal to the percentage of the (same) 
group scoring below the given essay score. Step-by-step pro- 
cedures are giVen rn Appendix A. ' \ 

Minimum Passing Scores : i 

The CSUP Norms Sample (see II above) will provide information to' 
determine the score on the essay part of the examination that 
corresponds to the passing point as defined in the announcement 
of the program "A Head-Start at Colleg-e" - i.e., performance at ^ 
the level of the average "C* student finishing the college English 
course. , 

The minimum ^passing score corresponding to the average "C" stu- 
dent's performance is to be determined in terms of the combined 
objective and essay scores. Setting a single cut-off point on 
the Composite score allows for freely compensating part-scores; 
i.e., a very poor performance on the essay part can be compensated 
for b^r a very good performance on the objective part, and vice^ 
versa. It has been decided that limits should be placed on this 
compensation. To ^ass the test a candidate must earn a Composite 
score equal to or greater than ^the Minimum passing score (middle 
"C") and he must earn part scores (objective and essay) equal to 
or greater than certain minimum scores (to be determined). 

As was pointed out at the April l8th meeting, it -\70uld be very 
desirable from a psychometric point-of-view to have ^both objec- 
tive and essay scores available for the CSUC Norms Sample. In 
that^ case minimum passing scores could be derived directly for 
both parts and the total. Because of the high costs in testing 
time and in, money, only the essay part will be administered to 
the CSUC Norms Sample. .Minimum passing scores for the objective 
part a/id the total test can be derived from other data which will 
be available. Two alternatives exist for estimating the level 
of perft>rmance of an,^ average "C" student on the objective part: 

1) The equating method described. in Section III establishes a 
' . relationship between object-ive and essay sc.ores in a way 
th^at allows dire ct' compari son . For example, an objective 
score of U9 represents the sanie level of performance as an 
^ essay score of li9. Therefore, the CSUC Norms Sample essay 
score distribution could be used to establish two "cutting- 
scores"; a)' ^he score corresi^onding to the level of per- 
formance, of an average "C" student which eventually would 
determine the minimum passing score on the 'total test, and 
b ) ,the minimum a'cceptable score on the ess ay part. Since 
the objective scores Represent the same level of performance 
as the corresponding essay scores , it seems reasonable -to 
use the same cutting scores Tor the objective part as is 
determined for the essay part. Stcp-by-step procedures are 
given in Appendix B-. 
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2) An alternative for estimating minimum passing points foi: 

the objective part involves the use of the CLEP Norms Sample 
and/or the Texas Norms Sample. Each of these groups con- 
sists of college students enrolled ,in "appropriate'^ English 
courses and each group took the CLE^P objective test. Data 
are available to fndlcate the level of performance of an 
average "C" student. However, average "C" is defi'-n-ed in 
terms of grading standards at non-CSUC institutions and 
"appropriate" courses- were selected by non-CSUC staff to ' . ' 
match the description of the content of the CLEP examination. 
The CLEP Norms Sample consists of students from 1? colleges 
representing a wide range of types of institution and geo.- 
gr aphi cal locations. The table below shows the mean CLEP 
score for the vario^us course g-rade categories for each norms 
group. , 

' ' NORMS SAMPLE 

4 < ■ 

CLEP . TEXAS 

Mean Score for: 

" ' "A" stildents ^ 6l 6l 

"B" students, ' 55 55 

' "C" students . ^9 ^9- 

"D" students ■ - 

"F" students • , JjO 50 

* 

^ Total Groups ^ 50 52 

The remarkable similarity of the Texas and National ^>ata is 
supportive of the use of national norms data to establish 
minimum passing scores. However, in using national norms 
data to establish minimum passing scores for the objective 
part, it is assumed that collective standards for course 
grades among the 17 colleges aVe similar to standards for 
the CSUC and that the relationship of CLEP scores to cours'e 
grades is similar in the two situations. 

Having two methods available for estimating objective cut- 
off scoreiP, one method can be used to check on the other, 
or modify the other if subs tan ti al di f f erences occur . I 
re commend that the equating method be used to determine tjie 
minimum passing scores and the CLEP Norms- data be used to • 
check, or modify, the results. I think it is desirable to 
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derive the minimum passing scores directly from Ca^lifornia 
^Lata. Appendix B describes the procedure in detail, ! 

I 

V . Checks on Reading Reliability : ; 

Three types of checking and control will *be used with respect 
to reading' reli ability . 

1) Table Leaders will continuously spot check the grading of 
readers at their table aifd Question Leaders will continu- ' 
iously monitor the grading of all the tables reading their 
question. These checks are informal in that no record will 
be kept but differences between the judgluents of reader and 
table leader or question leader will be resolve^d on-the-spot. 
This procedure is intended to maintain grading standards over 
the duration of the reading. i 

i -> 

2) When the grading is comjpleted fo^r a , paper, the cover (car- 
bon) strip is removed and the whole set of four grades is 
visible. The aide removing the cover will inspect the grades 
for consistency. The grades for first and second readings 

of the same essay ought to agree, within reasonable limits. 
There is no reason to expect grades for different essays to 
be the s^me\ so no checking is needed between the grades for 
.the first?^^§say and grades for the second essay. For con- 
^ venience in referring to each of the four grades^, let: * 

Qll denote the grade for Question 1 , first reading 

Q12 denote the grade for Question 1, second reading 

Q21 denote the grade for Question 2, first reading ^ 

Q22 .denote the grade for Question 2, second reading 

Then Qll and Q12 should be" compared and Q21 and Q2'S should 
be compared, but no checking is needed^ for other possible 
p ai r s , 

The difference between grades for two re agings that is to 
be accepted is fairly abitrary. It is probably not feasible 
to try to resolve differences of l^^.or 2 points, but differences 
of 3 or" more points probably ought to be resolved. 

The procedure for resolving, differences of 3 or more points 
should be estab'lished prior to the reading. It is suggested 
that the aide return' the paper with discrepant grades to 
the Question Leader. The Question Leader will read the essay 
himself. or assign it to* a third reader to try to determine ^ 
uhich grade is in inaccord with the^ standards . ThQ Question 
, Leader will .then discuss the ^ discrepancy with the reader and/ 

^or Table Leader to maintain consistency of grading standards 
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as well as to try to resolve the difference in grades, 
seems inappropriate for anyone to change a grade without 
the consent of the reader who assigned it. i 



It 



An alternate procedure would be to return the paper to the 
Question Leader, obtain a third reading, and use all three 
grades. This procedure would be faster b\it does not have , 
the beneficial effects on maintaining grading standards and 
allows "incorrect" grades to eater intc5 a student^^s total 
. score. The essay total should be adjusted to include only 
2/3 of the sum of the t'hree grades, rounded to the nearest 
whole number, plus the grades for the other question to ' 
keep the essay total on the same scale as regular papers. 

3) The thiird type of check on reading reliability involves 

obtaining correJ.ations between grades for first and second 

readings. This work can be don'e by the statistical aide 

following the recording of each day's produption. 

» 

3 

The reading reliability for question 1 is 'the correlation 
between grades Qll and Q12, The reading reliability , for 
question 2 is the correlation between grades Q21 and Q22. 



VI. ' Validity : 



'The college professors of the students in the CSUC Norms Sample 
have been asked to supply. two measures of performance in the 
college English course for each student: 

1) the letter-grade in the course 

2 ) 'a ca^tegorical ranking in a forced distribution 

Corelations between essay total scores'* and each of the two per- 
formance measures can be obtained as the data become available 
to obtain validity .coefi ci ents for the essay scopes. 
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CSUC ENGLISH EQUIVALENCY TEST 
STATISTICAL PROCEDURES 

APPENDIX A 



PROCEDURES FOR EQUIPERCENTILE EQUATING 



May, 1973 



William Cowell 



1, Before the reading begins the essay booklets from the regular 
candidate group should be put*in a random order. It is partic- 
ularly important that the Equating Sample be a random sample, of 
the total group. If a candidate used more than one booklet, 
the booklets should oe* securely fastened together to withstand 
many handlings during the reading. 

2. About 500 'essay booklets selected at random from the regular 
candidate group should be mixed yi.tli the booklets from the CSUC 
Norms Sample. These thousand (a'pprox.) booklets should be read 
first to enable the statistical^ vork of equating and norming to 
begin as early as possible. 

3,. As the reading is completed on each batch of booklets, the gr^ade 
cover (carbon) strips are removed and the aide should check for 
consistency , of grades between first and second readings. Grade 
discrepancies should be resolved as- quickly as possible and the 
paper returned to the aide. Booklets for the CSUC Norms Sample 
should be separated from those for the Equating Sample. A 
special code for the Norms Sample will identify those booklets. 
(See Appendix B) 

1|. As the completed booklets are sorted, the information from the 
Equating Sample booklets should be recorded on the form: 

ROSTER OF SCORE INFORMATION 
CSUC ENGLISH EQUIVALENCY TEST 
EQUATING SAMPLE 

The columns headed "Converted" and "Composite" will be filled 
in at a subsequent step. The Objective Score will be copied 
onto this form from the alphabetical roster of objective score 
information supplied by ETS . The information can be recorded 
in the order that the 'booklets are r^ceivedidt is not necessary 
to alphabetize this roster. If a candidate has an essay book- 
let but is not listed on' the objective score roster, remove the 
booklet from the Equating Sample. 
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5. After the information has been recorded on the roster, the 
information should be «:checked by another person and red dots 
placed beside the Objective Score 'and Essay Total Score to 
indicate that these data have been checked and found to be 
correct. If a dlscreponcy is found, cross-out the incor\fect 
figure and write the correct figure in red, then have it^ re- 
chec-ked. 

6. Sort the booklets into 25 piles dejpending on the Essay Total - 
Score (0-2h). The number of booklets in each pile will give 
the frequencies for the "Distribution o.f Essay Grades", 

7. Aft.er all the booklets for the Equating Sample have been read 
and the information recorded on the roster, score distributions 
for the Objective Scores and Essay Total Score will be 
obtained. Use the forms "DISTRIBUTION OF OBJECTIVE SCORES" and 
"DISTRIBUTION OF ESSAY TOTAL SCORES" to make a tally of the 
scores. One person can read scores from -the roster while another 
tallies the scores on the distribution. Then, for the objective 
scores, reverse the jobs and make another tally to check the 
first. For the essay scores, a count of the booklets in each 
pile can be usea to check the tallies. (See step 6 above) It 

is very important that these distribtuions for the Equ^ating 
Sample; be accurate since they will determine grades for the total 
group. ^' 

8. When an accurate tally has been obtained, write the frequencies 
(counts) on another copy of the form. Then fill in the 
"Cumulative Frequency" column, accumulating frequencies from the 

O bottom to the top. The figures in the "Frequency" and "Cumula- 
tive Frequency" columns should be the same for the lowest sco.^e, 
and the cumulative frequency for the highest score should be 
the total number of candidates in the sample . 

9. Have the "Frequency" and "Cumulative Frequency" data checked. 

10. Fill in the "Percent Below" column. The percent below for any 
given score is the cumulative frequency of the next lower score 
divided by the total number of candidates (rounded to the nearest 
tenth percent). For example, to find the percent below for a j 
score of 20, find the cumulative frequency for score-potnt 19 

and divide that number by the total number of candidates in the 
sample. Then more trhe decimal point two places to the right 
(equivalent to multiplying by 100) and round to the nearest 
tenth (e.g. , 1^8. Y) • 

11. Have the percent below Calculations checked. Place a red dot 
beside the percents as they are checked to indicate they have 
been checked and found correct. 
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12. Plot the CLEP objective scores against their percent belov 
values on arithmetic probability paper. Check the plotting, 
then ink in the points. Draw a smooth curve through the points 
using a French-curve. (If a computer program for curve-fitting 
can be -accessed via the ETS remote terminal, the computer pro-- 

^gram may be used to check on the graphical procedure..) 

13. For each Essay' Total Score, the corresponding CLEP Scaled Score 
will be found by using the graph obtained in step 12 . For each' 
Essay Total Score, . 

c 

a) find the percent below corresponding to that score on the 
"Distribution of Essay Total Scores" 

b) locate the corresponding percent below ,a^long the horizontal 
axis of the graph 

c) read the corresponding CEEP score "from the vertical, axis (to 
the nearest integer value) 

d) record equivalent Essay Total and CLEP Scaled Scores on 'the 
"SCORE CO«YERSJON TABLE". 

1*1. Have ste'p 13 checked. 

15. Record th^. .cjonver ted scores on the "Roster of Score Information". 

16. Add the objective and the converted essay score to obtain the 
Composite Scores. 

17. Have steps 15 and l6 cliecked. ' 

18. Tally the composite scores and. obtain the "DISTRIBUTION OF 
COW^IPOSITE SCORES" as in steps T-H. 

< 

19. Obtain the correlation coefficient between objective and converted 
essay scores (including means and standard deviations of each 

J score distribution). 
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CSUC EIIGilSll EQUIVALENCY TEST 
STATISTICAL PROCEDURES 

APPEHDIX B 

PROCEDURES FOR ESTABLISHING MINIMUM PASSING SCORES 



May, 1973 



William Cowell 



1. The essay booklets from the CSUC Norms Sample should be mixed 
with about 500 booklets selected at random from the regular 
candldaire group. These thousand (approx.) booklets should be 
read first, 

2. As, the reading is completed on each, batch of booklets and the 
consistency of grading is checked, booklets for the CSUC Norms 
Sample should be separated from those for the Equating Sample. 
(See Appendix A, steps 1-3)' 

3. As the booklets are sorted, the in,f6rmation from the CSOC Norms 
Sample booklets should be recorded on the form: 

ROSTER OF SCORE INFORMATION 
CSUC NORMS SAMPLE 



obtained, from the rosters 
Essay grades; and Essay 
course grade and/or rank index 



Cou^rse ^grades and rank index can be 
supplied by the college professors. 

Total should be recorded even if coi 

are not available. The column headed "Converted" will be filled 
in later. The information can be recorded^6n this roster in 
the order that the booklets are received; it is not necessary to 
alphabetize. 

Aft-er the information has been recorded o^ the roster, the infor- 
mation should be checked by another p^rsoh'. Red dots should be 
placed beside the essay total score and th*e course grade to . 
indicate that these data have been checked and found to be ^ 
correct. If a discrepancy is found, cross out the incorrect 
figure in red, then have it re-chVcked . 

Sort the booklets for the. CSUC Norms Sample into six piles de- 
pending on the code for Course Grade (A, B, C, D, F, and No Grade 
Reported ) . . 
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6. *After all the booklets for the Norms Sample have been read and 
the information recorded on the roster, six distributions of 
essay total scores should be constructed^ one distribution for 
each of the grade categories A, B, C, D, F,-an.d Uo Grade Re- ^ 
port-ed. Steps T-11 of Appendix A describe' the procedures. 

7. After these six distributions have been obtained and checked, a 
distribution of essay total scores for the entire CSUC Norms ^ 

• ' gro.up can be obtained by adding ^the frequencies across. the six ^ 
distributions (for each essay score). 

8. Find the mean and standard deviation of each of the seven dis- 
tributions . 

9. * The mean of the distribution of essay total scores for the grade 

' category "C" is t*he basis of- the minimum passing score. Use the 
graph constructed in step 12 of Appendix A to find the CLE? 
Scaled Score (Converted Sc-ore) corresponding to the me^n "C" 
grade. Multiply this score by 2 and round to the ne'arest in- 
teger value. This is the minimum passing score, using Method 1 
as described in "Statistical Procedures". 

10. To compare the results of the two methods suggested in "Statis- 
tical Procedures" for establishing the minimuijt passing score,, 
add k8.3h to the CLEP Scaled Score corres|)oV]da'ftg to the mean 
"C" grade. (The value liS.b'^'is the mean CLEP score for the 
CLEP National Norms Sample.) 

11. The minimum passing scores derived by each of these methods 
(steps 9 and 10) should be reported to the I>irector of the 
Reading for final determination of the minimum passing composite 
score . 

12. The minimum passing parl--scores must also be determined. This 
score should be set to minimize the number of classification 
errors (A, B, and C students who fail and number of D and F 
students who pass). U'o determine this point for the essay scores, 
construct two additional frequency distributions; one for the^ 
combined A, B, and C cou;rso-grade groups and one for the combined 
D and F groups. (Only the frequency columns need be filled in.) 
Inspection of these frequency columns will suggest one or more 
possible cutting points. The score which minimizes the number of 
classification errors should be selected- The Director of the 
Reading will make the final decision on this minimum passing 
part-s cor e's . 

13. After the "Score Conversion Table" has been completed (see ^ 
Appendix A), the Converted Essay Scores can be, added to the 
"Roster of Score Information". ^ ^ 
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Name 


Soc. Sec. No. 


Obj. 

Sco.re 


Essay Grades 


Composite 
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* Q21 


Q22 


Total 
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Name 


Soc, Sec, No. 


Course 
Grade 


Rank 
Index 


Essay Grades 


Qll 


'Q12 


Q21 


.Q22 


Total 


Convert e'l^ 
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DISTRIBUTION OF OBJECTIVE ^.SCORES 
" EQUATING SAf JPLE J 
• . May, 1573 ] 



Objective 
Score 


Frequency 


uutnuiacxve 
Frequency 


f \ ■y* /"s ^ t\ 1 • 

1 JLiI,CCIlU 

Re low 


VJU'J UC-;tiVc 

Score 


Frequency 


CiirRula tive 
Frequency 


Percent 
Below 


*80 
















79 


— 






49 


' 78 








•48 








. 77 








.47 








76 








46* 






,„„. f ...... 1 


75 








45 . 














• 44 









73 








■43 








72 








42 ' 








71 








41 








70 








40 








69 
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39 






68 


< 


38 








67 









37_; 

_36 









_ 66 










65 









35 
34 
33 




64* 
63 








62 




• 




32 








61 


— 


. 




31 

30 








60 


* " 








59>. 








29 






58 ' 








28 

27_ • 

26 ' 

25 

24 

23 






57 
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_ ^ _ . 
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Essay Total 
Score 


Frequency 


Cumulative 
Frequency 


Percent 
Below 


' 24 








23 








22 








21 








20 









19 




^ ^ : 




18 








17 . 








16 









15 








14 ^ 






13 








i2 . 








11 
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Composite 




Cumula tilvc 


Peizcont 


Scores 


FrcJqueiicy 


Frequency 


Below 


160 








155 - 159 


» 






150 -^154 








145 - 149 








140 - 144 








135 - 139 








1-30 -• 134 








125 - 129 








12.0 - 124 








115 - 119 








110 - 314 








105 - 109 


- --- -. ■-, 






■ 100 - 104 


4 






95 - 99 








90 - 94 








85 - 89 




* 




■80 - 84 








j5 - 79, _ 








70 - 74 




• 




65 - 69 








60 - 64' 








55 - 59 








_ 50 .-.54 








. ^<5 -_4? .J 








40 ~ 44' 








35 - -39 








. '30.- 34 
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20 - 24 








15 - 1? 








10 - 14 








5 - 9 _ 








0 - 4 
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THE CALIF0R>;NIA StATE UNIVERSITY 
English Equivalency test 
' STATi'STICAL REPORT • 

V William Cowell 



The California State University English 'Equivalency Test was 
adninlstered to 4,071 students on May 12 , 1973. The test consists 
■of two ninety-nihute-parts; an objective (nultlple-choice) part, and 
an essay part. The objective part is the Subject Examination 
"Analysis and Interpretation of Literature" of the College-Level' 
Examination Program. (CLE?). The essay p.art consists of two questions 
prepared and graded by the Englis-h^ processors of the California S-tate 
University . and Co-lleges (CSUC). 



II. Scores . 

(1) ObjecUive Score , The Educational Testing Service scored the 
objective part and Reported scores to the students , and the. CSUC 
English Tes*t' Center on the CLE? score-scale of 20 to 80. 

(2) 'Essa y Question Scoxes. . Each of uhe two essays uas graded by two 
readers whb, assigned scores on a scale of. I to 6. Zero denotes that 
the student did not* actenpt an ar.sver to the question. 

(3) Raw Essay Total Score . The four essay question scores were' added 
to obtain the raw essAv tOwai score. The.e'ssay total scores range 

.from 0 to 24. Scores of 0 ro 3 are possible only if the stud-^nt did 
n'oT an^swe-r one ,ox,b_oth c;uestions. 

^ ^ ^~ '~ — 

(4) Converted Essay Scar e. The raw essay total scores we^re t-r.ans-.. 
formed to the sane scale as the objective scores ,tc permit addition 

,of objective and essay scores in a neaningful way. The^ technique 
" ' for converting the- scores is described below (Section^ 

<» 

(5) Composite Scores . Composite scores for the whole examination 
were obtained by adding the Converted Essay Scores and the Objective 
Scores . . 

j.II. C andidate Groups and ><Qrnis Sazples . 

(1) ' Total Group . The to-Cal group consists of the 4,071 students who 
took the English' Equivalency Test in Ma-y- 1973 . 

(2) Equating -^rSample . The equating sample consists of the first 53^6 
papers completed at the reading session. Essay books were scrambled 
be'fore the reading to essentially randomize ehe order. The sample 
is not significantly d'ifferent fron the total group with respect to 
the mean and standard deviation ,of the objective scores. 
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Objective Scores 

Mean S , D , 

Total Group 47,45 8.85 

Equatiag Sample 47.63 8.75 

\ 

(3) CLEP Norms Sample\ The CLEP Norms Sample consists of the 541 
college students who participated in the 1964 norming administration 
of the CLE-P Analysis ar^^ Interpretation of Literature Examination. 

(4) Texas Norms Sample . The Texas Nornc Sample consists of the 188 
Srtudents who participated in che 1970 validity study of the CLEP 
examination at the University of Texas. 

' / ^ ^ 

(5) * sCSUC Norms Sample . The essay part of the English Equivalency 
Test was administered to a sample of CSUC students^ completing a year 
of college English in an attempt to link performance on the test to 
performance in their course (as measured by their grade in the 
course). It had been planned to use the scores of the CSUC Norms 
Sample to' establish the minimum passing score for the essay part of 
the examination,^ , Since fewer- thafn half the expected numbar of papers 
we.re received in time for the reading and because the scores were* 
unreasonably low, it was decided to base the minimum passing scores 
on other criteria (see section VI, below). A summary of,* the score 
information for the' Norms Sample is shown in Table Zj Although the 
data, may not be very reliable, they do provide some indication of 
the relationship between college grades and grades on. the essay part 
of the exam. ^ ^ *, ' 

Table 2 

^ ' Summa-ry- Data for CSUC Norms Sample 

Raw Essay Total Scores 



_C o u rs e 
Grade 




Mean 


Standard 
D.e.Yiat ion 


A 


26 


13.5 


4.0 


B 


75 


12.9 


3.0 


C 


98 


10. '6 


3.5 


D 


33 


10. 1 


3.2 


F 


10 


7.8 


2.4 


Total 


242 


11.5 


3.7 



IV . Equating the Essay and Objective Scores . 

"Equatiug" scores means making them equivalent with respect to 
some characteristic. Here, the essay scores were transformed to the 
same scale (20 to 80) as the objective scores so that the transforme^d 
essay scores could be added to the ^objective scores to obtain a com- 
posite score in which the two parts would be equally weighted, Kad 
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the raw essay scores been added to the objective s^£>res, the essay 
part would have had a weight of only 1/4 of the total. Also; tha 
transformation was done i^n such a way that the converted essay sco.res 
can be directly compared with objective scores. That is, a converted 
'essay score of 4*, for example, represents the same level of perfor- 
mance as an objective score of 48, 

Of severaj. acceptable methods of equating which are available , 
t'he equipercentile (curvilinear) method was selected as the most 
appropriate. The converted score corresponding to each, raw essay 
score' is that score in the distribution of objective (CLEP) scores 
such *that the percent of the group scoring below thac objective 

3core is equal to the percent of the group scoring below the given 
essay score. For example, to find the converted score corresponding 
to a raw essay score of 15, we observe 'from the distribution of essay 
scores that about 68 percent of the group scored below 15 on the 
essay part. From the distribution of objective scores it is deter- 
mined that a score of 53 is that score be.low which 68 percent of the 
group scored. So a raw score of 15 is equivalent to a scaled score 
of 53. Similarly, a scaled score Is found for each raw essay score. 
Table. <l shows the converted score corresponding to each raw essay 
score. 

V. Technical Notes about the Equating . 

(1) The equating was done using data from the Equating Sam'ple des- 
cribed in 111(2) above. The sample w^s used instead of the total 
group so that the statistical work could proceed along with the essay 
reading. Score conversions and distributions could be prepared for 
the sample data in time for a meeting with Question Leaders at the 
reading. The time and cost of equating was also, reduced by using a 
sample rather than the total group. Scrambling the essay booklets 
before the reading provided a sample which was representative of the 
total group. * 

(2) A method of graphical smoothing, described, in "Educational 
Measurement"^ ^ used to interpolate between percentile ranks in 
the distribution of objective scores. 

(3) Altho'ugh the equipercentile method was selected because it allows 
for a non-linear relationship between objective and essay scores, 
,the correlation be'tween raw essay and converted essay scores is .999 . 
This extremely high coefficient of linear relationship indicates 
that the simpler linear methods^ of equating could have been used 
with essentially the same results as the equipercentile method. 



1. Angoff, W. H.; "Scales, Norms, and Equivalent Scores" in 
R. L. Thorn dike (Ed,) Educational Measurement , 2 rid Ed , Washington , 
D. C.: American Council on Education, 1971, 563-590. 

• 2* Ibid . , page 572. 
3. Ibid . , page 513. 
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SCORE CONVERSION TABLE 
-CSUC ENGLISH EQUIVALENCY TEST 
May, 1973 

Essay Total S,core - Converted Score 



24 


- 76 


23 


- 73 


2 2 


21 


- by 


20 


- D / 


19 


~ 64 


18 


62 


17 




16 


- DO 


lo 




14 


- 51 


13 


- 48 


12 


- 46 


I'l 


- 43 


10 


- 41 


9 


- 38 


8 


- 35 


7 


- 33 


6 


- 31 


5 


- 28 


4 


r 26 


3 


- 24 


2 


- 22 


1 


- 20 


0 


- 18 



Minimum Passing Scores . 

The ninimum passing scpre for -the total test was determined by- 
setting a passing point for each part and adding the scores. For 
both the CLEP Norns Sample and the Texas Norms Sample, described in^^ ■ 
paragraphs III (3) and III (4) above, the average CLEP.scoize for C 
students was 49. It was decided that 49 should be the minimum passing 
level for the objective part. "The nininun passing score for the 
essay p'^irt was set at 14, which converts to a scaLe'd score of 51.. A 
composite score of 100 (49 objective + 51 essay) was thus determined 
to be the nininum passing score on the examination. 

Setting a single cut-off point on the Coriposite Score allows 
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icmpsn sating part-scores; 



a very poor performance 
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on the essay part can be conpensated for by a very good performance 
on the objective part, and vice versa. It was decided that limits 
should be placed on this compensation. In addition to yarning a 
composite score" of 100 or more, a student had to achieve a minimum 
score on each part. These minima were set at 45 for the objective 
part and 48 for the essay part. (A converted score of'48 corres- 
ponds to a raw essay score of 13.) A student who scored 12 or lower 
on the essay part could not pass the ^exam no matter* how high the 
objective score. A raw essay score of 12 represents tlie score level 
of uniform agreement among four readers that the essay performance 
is unsatisfactory. An objective score of 45 was set as^ the cat-off 
p'oint because in both the CLEP Norms Sample and the Texas Norms 
Sample a score of 45 minimized the number of A, B, and C students 
who would fail and, the number of D and F students who would pass. 

* VII. Number and Percent Passing . 

Of the 4,071 students who toolc the test,^ about 33 percent 
(i.e. 1, 343 students)^ passed and are eligible to receive six semester 
hours of college credit. From t-he distribution of scores on the ob- 
jective part, it can be determined that about'43 percent (i.e. 1.733 
students) "passed" the objective half of the test that^is, about 
43 percent of the total group earned scores of 49 or higher on the 
abjective part,. Therefore, had only the objective part of the test 
been given andy^ame ziininum score *been used, an additional 10 percent 
would have passed. .These students failed because their essay scores 
were un satisfactory* 

Corresponding data based on the distribution of essay scores 
are not yet available' for the total group. Estimates based on data 
from the Equating Sample will be discussed* in the following section. 

VIII. Estimate's Based on Data from the Equating, Sample . 

Estimates of t*he characteristics of the total group can be made 
from data available for the Equating Sample. It has been pointed 
out in p.aragraph III (2) that the mean and standard deviation of ob- 
jective scores for the sample are very nearly'equal to those pf the 
total group. It is not clear, however, that the essay scores for .the 
s'^ampla are representative of the total. group. It is" not uncommon, 
for example, to find that grading tends to become more severe near 
the end 6f an essay reading session. Until final essay results are 
tabulated, a direct check on essay data is not possible. As an in- 
direct check on the similarity pf the sample and total group, the 
percents passing the entire exam (objective plus essay) were compared. 
Whereas only 33.0 percent of the total group passed the entire exam, 
37.5 percent of 'the .Equating Sample passed. This rather large dif- 
ference suggests that the following data be used with caution, 
allowing for the possibility of substantial changes when the final * 
distributions are available. When actual data become available, 
these estimates should be disregarded. 
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(1) Tor the Equat ing' Samp 1 e , 43..1 percent ''passed** the ess^y part 
of the, exam - that is, 43.1 perc.ent of the sample earned scores of 
51 or higher on the. essay part. The difference between 43,1 percent 
"passing'' the essay part and 37,5 percent passing the entire exam 
(i.e. 5.6*percent) is the percent of the'*sample who failed because 

of an unsatisfactory performance on the objective part. Had only the 
essay part of 'the. exam been given and the same minimum passing score 
(i.e.. 51) bee.n used, an additional 5,6 percent of the sample would 
hav^e passed. This estimate is probably a fairly reliable estimate 
for- the total group because the sample and total group are very 
nearly equal on objectives-part performance. Projected to the total 
•'group of 4,071 students, an additional 225 to^ 265 students would 
have passed if only the essay part been given. 

(2) "0f the 536 'students in ttie Equating Sample, 218 had Composite 
Scores of 100 or higher, i.e. they had "passing" Gonposite^ Scores. 
Of these 218 students, 201 passed the entire exam'. The remaining 17 

* st^udents (about 3 percent) failed because of the limits placed on 

compensation between the part scores. -Of these 17 students, 11 

failed because their essay scores were lower than 48* and 6 failed 

because their objective scores were lower than 45**. Projected to 

the total group, it can be" estimated that only 1 or 2 percent (i.e. 

40 to 80 students) failed because their objectives scores were below' 

45 even though they earned Composite Scores of 100 or more. An 

estimated 2 to 6 percent (i.e. 80 to 240 stude'nts) failed because 

their essay scores were lower than 48 even though their Composite 

Scores were l6o or higher. 

* • 

IX. Correlatrons among the Parts . 

Correlations among the objective. Raw Essay Total, Converted 
Essay Total, and Composite Scores were obtained from data available 
for the Equat ing^ Samp le . If correlation tables become available for 
the total group, those tables should replace these but thes6 corre- ' 
latlons should be fairly reliable estimates of the corresponding 
total group data. 

The correlation of ,547 between the objective and essay part 
scores* seems about ideal for this type of examination. It is high 
enough to show that *the parts are testing related abilities and con- 
sequently that combining the par^t scores to obtain a single composite 
score is justified. How.ever, the correlation is low enough to justify 
the use of both parts; that is, each part is contributing some unique- 
component of its own. * * 

The co,rrelation of . 999 between the raw and converted essay 
scores indicates a strong linear relationship between the scores. 



*0f these eleven students, eight had scores qf 46 and three had 
scores of 41 (i.e. r^w essay scores of 12 and 10. respectively*) 

**0f these six students, three had scores of 44 and one each had 
scores of 43, 42, and 40. 
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Table 3 

Correlations among Part Scores and Composite 
for. the Equating Sample 



Score 


Obj ective 


Ra.w 
Essay 


Convert ed 


Composite 


Objective ' 
Raw Essay Total 
Converted Essay 
Composite 


1.000 
0.547 
0.547 
0.877* 


0. 547 

1. 000 
0.999, 
0.876* 


0.547 

0. 999 

1. 000 

' 0.877* 


0.877* 
0.876* 
0.877* 
1.000 


*Spuriously high 


because the part 


is included in the 


total . 



As noted earli'er, a linear equating method could have been used in- 
stead of the more Complex curvilinear method with essentially the 
same results. 

* •* 
Reliab ility ♦ ^ . ^ * . „ * 

(1) Reliability of Dhe'^ Ob j ect ive Part . The CLEP Score Interpretation 
Guide^,reports that; the reliab iJLity-of the ''Analysis and Interpreta- 
tion of Literature" Examination is .88. 

(2) Reliability of the Es^say Part . "Probably the most relevant kind 
of reliability coefficient for an es^.ay. test would be "f orm~to~f brm" 
reliability; that ^is, the coefficient that^w.o.ald be-~obtained by 
constructing two equivalent forms of the essay part, (either of which 
coiild be used in the actual administration)-, administering tKem both 
to a sample , of students, arfd obtaining the correlation coef f icient?> 
between the scores on the two forms* Each student in tKe sample- 
would take both tests. Since this type of data is not available, ^ 
tKe reliability of the essay part must be estimated by-other methods. 
Two methods are used; one- method gives an upper limit, the bther 
gives a lower limit. We can be reasonably' certain the actual, re- 
liability lies somewhere- between these two values* 

(3) The Upper Limit . This method is based on the correlation 
between grades on the first arid s.econd readings of each essay ques- 
tion'. The correlation betx;een the grades for the first and second 
readings of Question 1 is about •66, and that of Question 2 is abQUt 
•72. These values are the reliabilities for a simple essay grade. 
That is, each of the grades for the two readings of Question 1 has 



4» , Score Interpretation Guide, College-Level 

Examinations Program , College Entrance Examination Board, Princeton, 
N. J. , 1967, page 31. ^ 
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a reading reliability of so6 and ea.ch grade for Question 2 has a 
reliability of .72. Since the actual final grade includes the 
grades for both readings, t.he reading reliability for each question' 
is** higher than tha^t of a single reading. The Spearm'an-Brown 
Prophecy Formula^ can be used to estimate the r-eliab i.li ty af the 
tota.r for each question* For Question 1, the reading reliability is' 
about .80 and for Question 2, ^bout .84. Thev Essay Total score is 
the sum of grades for all four readings and the estimated reading 
reliability for the whole essay part is .90. 

a 

Sinc"^e the unreliability due po grading is only one of several 
sources of unreliability contained in the scores, the reading re- 
liability is alwa;^s greater tfian the score reliability and is an 
upper limit "of score reliability. The reliability of the essay part' 
would not be greater than .90.- 

(4) The Lower Limit . The second zethod o.f, estimtiting score relia- 
bilit.y is based ori the correlation between the scores for Question 
1 and Question 2. This is ,ari approximation to the f o^m-to-f orm ' 
reliability, usring -,each question as i^ it were a one-question testv " 
To the e':xt.ent that the different c,ue'stions oh one for'm are. .designed 
to measure different abilities within the general ' domain covered hy 
the tes.t, the questions within one form laay not be equivalent in the^* 
same sense, as t"wo different fori:s are equivalent. The-refore, the * 
coefficients obtained in this way^ r:ay be expected to underestimate 
the actual score reliability and nay be regarded as 'a lower limit 

of the reliability of the essay section. ^ ' ' " . 

Based on the data from' thft Equating Sample, the correlation ' 
between the grades *for Questions 1 and 2 is about .-41. * This is an 
estimate of the reliability cf each "one-question" test. The actual 
tests consists of. both question,s so the reliab-ility is higher. The 
Spearman-Brown Formula is used uo estimate the reliability of the 
entire essay part and gives an estimate of about .58. The actual 
score reliability is probably betvaen .58 and .90. 

(5) Reliability of the Composite Scores . Based on reliability co- ' 
efficients of .88 for the "ob j ec t ive part and between .58 and .90 
for the essay part, it is estimated that the^ r eliab ili ty of the 
Composite Score is between .83 and .93. ' 

Validity . 

4 

* ' 6 * 

The CLEF Score Interpretation Guide reports correlatio^ri of .42 
and' .54 between scores on the "Analysis and Interpretation of 
Literature" examination and course grades before and after the final 



5* Op. ci t.," Educational Measurement ,* pages 71-72. 

fc. ' " ' " 

6. Op . cit . , pages 33-34. 
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exanlhation^ respectively. The final cpurse grades may or may not 
include the score on the CLEP exan and the grades were given at a 
variety of colleges wit;h differing grading systems. 

Using^ data from the* CSUC Norns Sample, a correlation coefficient 
of .39 between Essay Total Scor'es and course grades was obtained. 
The cautions suggested in the CLEF^ Guide> and . the cautions noted in 
section III (5) above should be considered- in interpreting this ^ 
correlation. 
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February, 1974 



I. The Test. 



The California 'State University 
ENGLISH EQUIVALENCY TEST 

STATISTICAL REPORT 

• ■ William R. Cowell 



The California State University English Equivalency Test was 
administered to 4.071 students on. May 12, 1973. The test consists of two 
ninety-minute parts; an objective (multiple-choice) part and an essay part-. 
The objective part is the Subject Examination "Analysis and Interpretation 
of Literature" of the College-Level Examination Program (CLEP). The essay 
part consists of - two questions prepared and graded by the English profes- 
sors of the California State University and Colleges (CSUC). 

II. Scores . 

(1) Objec tive Score . 'The Educatibnal Testing Service scored the objective 
part and reported scores ,to the students, and the CSUC English Test Center 
on the CLEP score-scale of 20 to 80. 

(2) Essay Question Scores . Each of the two' essays was graded by two 
readers who assigned scores- on a scale of 1 to 6. Zero was used to de- 
note that the 'student did not attempt to answer the question. 

(3) Raw Essay Total Score . The four essay question scores were added to 
obtain the raw essay total s.core. The essay total scores range from 0 to 

'24. 

* (4) Converted Essay Score . The raw essay total scores were transfdrmed to 
^ the scale as the objective scores to permit addition of objective and 

essay scores' in a meaningful way. The technique for converting the scores 
is described in Section .IV. 

(5) Composite Scores . Composite scores for the whole examination were 
obtained by adding the Converted Essay Scores and the Objective Scores. 

III. Candidate Groups and N6rms Samples . 

(1) T6tal Group . The total group consists of the 4,071 students who took 
the English Equivalency Test in May 1973. 

^2) Equating Sample . The equating 'sample consists of the first 536 papers 
completed' at the reading session. Essay books were scrambled before the 
reading to essentially randomize the order. The sample is not significantly 
different from the total group witl«respect to the mean and standard devia- 
tion of either the objecti,ve scores or the essay scores. 
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Table 1 

Comparison of Equating Sample and Total Group 



< 

Group 


Oblective Scores 


Essay Scores 


Mean 


S.D. 


Mean 


S.D. 


Total Group ' 
Equating Sample 


47.45 
47.63 


8.85 
8.75- 


47.92 
48.25 


8.40 
8.86 



(3) CLEP Norms Sample . The CLEP Norms Sample, consists of the 541 college 
students who participated in the 1964 norming' administration of the CLEF 
Analysis and Interpreta;t£on of Literature Ex^ination. 

(4) .. Texas Norms Sahpl^ . The Texas Norms Sample consists of ,the 188 
students who participated in the 1970 validity study of the CLEP examination 
at the University of Texas. 

(5) CSUC Norms Sample . The ^ ess ay part of the English Equivalency Test was 
administered to a sample of -CSUC students , completing a year of college 
English in an attempt to ^.ink performance on the test to performancTfe in 
their course (as measured by their grade in'the -course) . It had been planned 
to use the scores of the CSUC Norms Sample to establish the minimum passing 
score for the essay part of th^. examination. Since fewer than half of the 
expected number of papers were received in time for the. reading and because 
the scores were unreaspnably low, it was decided to base the minimum passing 
scores on other criteria (see Section VI) . A summary of the score Informa- 
tion for the Norms Sample is shown in Tab^e 2. These data provide some in- 
dication of the relationship between college grades and .grades on the essay 
part of the exam. 

Table 2 

Summary Data for CSUC Norms Sample 
Raw Essay Total Scores 



Course 
Grade 


N 


Mean 


Standard 
Deviation 


A 


. 26 


13.5 , 


4.0' 


B 


75 


12.9 


3.0 


C 


98 


10.6 


3.5 


D 


33 


10 .'1 


. 3.2 


F 


io 


7.8 


2.4 


Total 


242 


11.5 . 


/3.7 
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IV. Equating the Essay and Oble ctlve Scores. 

"Equating" scores means making them equivalent with respect to some . 
chatacteristic. Here, the essay scores were tran^ormed to the same scale . 
(-20 to 80) as the objective scores so that the transformed essay scores 
could be added to the objective scores to obtain a composite score in which 
the two parts would be equally weighted. Had the raw essay scores been 
• added to the objective scores, the essay part wtfuid have had a weight ot I/'* 
of the total. Also, the transformation was done in such a way that the 
converted essay scores can be directly compared with objective scores. That 
. Is, a converted essay score of 48, for example, represents the same level of 
performance as an objective score of 48. 

" Of several acceptable methods of equating which are available, the 
equipercentile (curvilinear) method was selected because the nature of the 
distribution of essay scores could not be predicted. [See V (3)]. The con- 
verted score corresponding to each of the raw essay scores was found by the 
procedure illustrated in the following example. To find the converted score 
corresponding to a raw essay scor^e of 15, for example, we observe from the 
distribution of essay scores that about 68 percent of the group had scores 
below 15. We then look down the "percent below" .column in the distribution 
of objective scores for the percent-below closest to 68. The objective score 
corresponding to that percent-below is designated as the scaled score for 
an essay score of 15. In this case, the objective score of 53 corresponds 
to 68 -percent below. So a raw essay score of 15 is equivalent to. a scaled 
score of 53. [See V (2)]. Similarily, a scaled score is found for each raw 
essay score. Table 3 shows the converted score corresponding to each raw - 
essay score. ^ 

V. Technical Notes. About the Equating . 

(1) The equating was done using data from the Equating Sample described in 
111(2) above. The sample was used instead of the total group so that the., 
statistical work could proceed along with the^essay reading; i.e., score 
conversions and distributions could be prepared for the sample data in 

time for a meeting with Question. Leaders at the reading. The cost of equat- 
ing was aiso reduced by using a sample rather than the total group. 
Scrambling the -essay booklets ^before the reading provided a sample which 
was representative of the total group. 

2 

(2) A metho4 of graphical smoothing, described. in "Educational Measurement" , 
was used to interpolate between percentile ranks in the distribution of 
objective scores. e 



1. .Angoff, W. H.; "Scales, Norms, and Equivalent Scores" in * 

R. L. Thorndike (Ed.) Educational Measurement , 2nd Ed., Washington, D. C. : 
American Council on Education, 1971, pp. 563-590. 

2. Ibid. , page 572. ' J 
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(3) Although the equlpercentile method was selected because it allows for 
a non-linear relati'onship between objective and essay scores, the correla- 
tion' between raw essay and converted essay scores is .999. This extremely 
"high coefficient- of linear r-elationship indicates that the simpler linear 
methods^ of equating could have been used with essentially the same results 
as the equlpercentile method. 



Table 3 

Score Conversion. Table 

CSUC English Equivalency Test 

May, 1973 

Essay Score Scale'd Score* 

24 . '76 

23 73 

22 71 

21 69 

20 67 

19 64 , 

18 62 

17 59 

16 56 

15 53 

14 51 

13 . 48 

-*>•- 12 ' 46 

■ 11 43 

10 41 , 

9 38 

8 35 

7 33 

6 , 31 

5 -28 

^4 ' . '26 

3 24 

2' 22 

1 20 

0 18 



3. Ibid . , page 513. 

i 

1G« ■ 
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VI. Minimum 'Passing Scores . 

The minimum passing- score for the total test was determined by setting 
a passing point for each part and adding _the scores. For both the CLEP 
Nanns Sample and the Texas Norms Sample, described In paragraphs III (3) and 
III (4) above, the^ average CLEP score for **C" students was 49. It was de- 
cided that 49 should be the minimum passj^ng level for the objective part. 
The minimum passing 9core for the essay p,art was set at 14, which converts 
to a scaled score of 51. A composite score of 100 (49 objective + 51 essay) 
was thus determined to be the minimum passing score on the examination. 

Setting a single cut-off point on the Composite Score allows for 
freely compensating part-scores; that Is, a very poor performance on the 
essay part can be compensated for by a very good performance on the objec- 
t tlve part, and. vlce~ versa. It was decided that limits should be placed on 
this compensation. In addition to earning, a composite score of 100 or more, 
a student had to achieve a minimum score on each part. These minima were 
set at 43 for the objective part and 48 for the essay part. A converted 
score of 48 corresponds to a raw essay score of 13. A student who scored 
12 or lower on the essay part could not pass the exam no matter how high 
the objective score. A raw essay score of 12 represents the score level of 
uniform agreement among four readers that the essay performance Is unsatis- 
factory. An objective score of 45 was set as the cut-off .point because In 
both the CLEP Norms Sample and the Texas Norms Sample a score of 45 mini- 
mized the number of A^ B« and C students who would fall and- the number of D 
and F students who would pass. 

VII. Number and Percent Passing . , » - . 

Of the 4,071 students who took the test, approximately 33 percent of 
' ^ them (1,362 stud ents) passed and are eligible to receive six semester-hours 
credit. From the distribution of scores on the objective part It can be 
determined that approximately 43 percent "passed" the objective half of the 
test - that Is, -they earned scores of 49 or higher on the objective part. 
Therefore, had only the objective part of the test been used, and the same 
minimum passing score established, an additional 10 percent would have passed. 
These students failed because their essay scores were unsatisfactory. 

Approximately 40 percent of the total group achieved "passing" essay 
scores - that Is, they earned s^cores of 51 or higher on the essay part of 
' the test.*' Therefore, had only the. essay part been used and the same pass- 
ing point established, an additional 7 percent would have passed. 

In addition to earning a total score of 100 or higher^ a student had to 
earn scores of at least 45 on the objective part and 48 on the essay part. 
Of the total group, 153 students achieved scores of 100 or more on the total 
te^t-^but did not pass because they did pot earn the minimum score on one of 
the parts. 
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VIII. Reliability . , . - 

« 

. (1) Reliability of the Ob.lectly^ Part . The CLEP Score Interpreta^tlon Guide 
reports that the reliability of the "Analysis and Interpretation of 
Literature" examination Is ..88. 

(2)' Reliability of the Essay Part . Probably the most relevant kind of re- 
liability coefficient for an essay test would be "form-to-form" reliability;' 
that is, the coefficient that would ^be obtained by constructing, two e.quival- 
ent forms of the essay part (either of which could be used in the actual 
administration), administering them both to a sample of students, and ob- 
taining the correlation coefficient between the scores on the two forms. 
Each student in the sample^ould take both tests. Since this type of data 
is not available, the reliability of the essay part must be estimated by 
other methods. Two*^methods are used; one method gives an upper limit, the 
other gives a lower limit. We can be reasonably certain the actual relia- 
^ billty lies somewhere^between these two values. 

<• (3) The Upper Limit > This method is based on the correlation between 
grades on the first and second readings of each essay question. The cor- 
.relation between the grades for the first and second readings o^ Question 1 
is about .66, 'and that, of Question 2 is about .72. These values are Che 
reliabilities for a single essay grade. That is, each of the grades for 
the two readings of Question 1 has a reading reliability of and each 
grade for Question 2 has a reliability ^ of .72. Since the actual final grade 
includes the grades for both .readings, the reading reliability for each 
question is higher than that of a single reading. The Spearman-Brown 
Prophecy Formula^ can be used to estimate the reliabilit'y of the total for 
each question. For Question 1, the reading reliability Is about .80 and 
for Question 2, about .84. The Essay Total scqre is the sum of grades for 

' all four readings and the estimated reading reliability for the whole essay 
part is .90. 

Since the unreliability due to grading is only one of several .sources 
of unreliability contained in the scores, the reading reliability is always 
greater than the score Reliability and Is an upper limit of %core rella- 
bility. The reliability of the essay part would not be greater than .90.* 

The Lower Limit. The second method of estimating score reliability is 
based on the correlation between the scores for Question 1 and Question 2. 
This is an approximation to the forra-to-form reliability using each question 
as if it were a one-.question test. To the extent that the different questions 
on one form are designed to measure different abilities within the general 
domain covered by the test, the questions within one form may not be 
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^' » Score Interpretation Guide, College-Level Examinations 

Program, College Entrance Examination Board, Princeton, N. J. 1967 
page 31. f • 

5. Op. cit .. Educational Measurement , pages* 71-72. 
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equivalent in the same sense that two different forms are equivalent. 
Therefore, the coefficients obtained in this way may be expected to . 
underestimate the actual score reliability and may be regarded as a lower 
limit of the reliability of the essay section. 

Based on the data from the Equating Sample, the correlation between 
the grades for Questions 1 and 2 is about .41. This is an estimate of the 
reliability of each "one-question" test. The actual test consists of both 
questions so the reliability is higher. The Spearman-Brown Formula is used 
to estimate the reliability of the entire essay part and gives an estimate 
of about . 58 . 

(5) Reliability of the Composite Scores . Based on a reliability co- 
efficient of .88 for the objective part and a lower limit of -58 andean 
upper limit of .90 for the essay part, it Is estimated that the reliability 
of the Composite Score Is between .83 and' .93. 

IX. Correlations Among the ^arts 



Table 4 

Correlations Among Part Scores and 
Composite for the Equating Sample' 
•N - 536 



Score 


Obiective 


Raw 
Essay 


Converted 


Composite 


Objective 
Raw Essay Total 
Converted Essay 
Composite 


1.000 
0.547 
0.547 
0.877* 


0.547 
1.000 
-0.999 
0.876* 


0 . 547 
0.99.9 
1.000 
0.877* 


0.877* 
0.876* 
0.877* 
1.000 



*Spurlously high because the part is included in the total. 



to 



Correlations -among the Objective, Raw Essay Total, Converted Essay 
Total, and Composite Scores were obtained from data available for the 
Equating Sample. The correlation between the objective and essay part- 
scores is 0.547. This correlation coefficient is an indiex of the extent 
which the objective and essay parts measure the same abilities- If both 
parts were perfectly reliable, the correlation coefficient would be higher; 
i.e., the correlation coefficient is less than 1.000 partly because the 
objective and essay tests measure different abilities and partly because 
they are less than perfect measuring instruments. The correlation 



6. Ibid ., pages 401-404. 
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coef§ioient can be adjusted to eliminate the effect of the unreliability 
of the tests. 7 Using .88 as the reliability of the objective part and a 
lower limit of .58 and an upper limit of .90 for the reliabiLity of the 
essay part, the corrected (disattenuated) correlation between the objec- 
tive and essay parts is estimate^! to be between .61 and .77. 

'The correlation between the objective and essay part-scores seems about 
ideal for this type of examination. It is high enough to show tha^ the 
parts are testing related abilities and consequently that combining the part 
scores to obtain a single composite score is justified. However, the corre- 
lation is low enough to justify the use of both parts; that is, each. part is 
cpntributing some unique component of its own. 

The correlation of .999 between the raw and converted essay scores 
indicates a strong linear relationship between the scores. As noted 
earlier/ a linea^r equating method could have been used instead of the mote 
compJlex durvilinear method with essentially the same results. 

X. Validity . ^ 

The CLEP Scor^ Interpretation Guide reports correlations of .42 and .54 
between scores on the "Analysis and Interpretation of Literature" examination 
and course grades before and after the final examination, respectively. The 
final course grades may or may not include the score on the CLEP exam and 
the grades, were given at a variety of colleges with differing* grading systems. 

Using data from the CSUG Norms Sample, a correlation coefficient of .39 
between Essay Total ScorM and course grades was .obtained. The cautions 
suggested in the CLEP Gu<fde and the cautions noted in Section III (5) above 
should be considered in interpreting this correlation. 

XI. Summary 

The California Sate University English Equivalency Test was adminis- 
tered to 4,071 students in May 1973. .The test consists of two parts; an 
objective (multiple-choice) part and an essay part. The essay part was also 
administered to ,a normirig sample of 242 CSUC students. Essay scores weie 
converted to the same scale as the objective section scores by the equi- 
percentile (curvilinear) method. Data suggests that simpler linear methods - 
could have been used with essentdally the same results. 

To pass the exain, a student had to earn a total score of 100 or higher 

and, in addition, earn scores of 45 or higher on the objective part and 48 

or higher on the essay part. Approximately 33 percent of the students 
passed the exam* 



7. Lord, F* M. and Novick, M. R. ; Statistical Theories of Mental Test 
Scores, Reading, Mass . ; , ,Addison-Wesley Publishing Company, 1968, pages 
115-118. 
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The correlttlon between the objective and essay part scores is about 
.55; high enough to show the parts 'are related and low enough to show each 
part contributes some unique component to the total score. The reliability 
of the objective part la about .88 and that of^the essay part la probably 
between .58 and .9%. The reliability of the composite (objective plus essay) 
la probably between .83 and .93. 

The correlation between part-scores and college course grades Is 
•approximately .40 for both the objective and essay parts. 
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PUBLICITY FOR 1973-74 ENGLISH EQUIVALENCY EXAMINATION 

Publicity pertaining to the statewide English Equivalency 
Examination has included both program information and editorial , 
comment/ In December 1972, William Trombley in the Los Angeles 
Times reported on the agreement reached between the Chancellor's 
Office and the California State University and College English 
faculties for the English Equivalency Examination, and on the- 
nature, scope and purpose of the proposed examination. During 
its development in early '1973, the project as well as the concept 
of Einglish Equiyalency Testing came under professional scrutiny 
in the Chronicle of Higher Education . 

As firm timetables were established for the testing phase 
of the program., news releases from the Chancellor's Office and 
each of the participating institlutions provided^ notice of test 
dates, sites and prerequisites for participation. More than 
seventy-five o.ewspapers th^roughout the state, including* those 
with local and statewide circulation, carried pretest stories 
about the project. The pretest publicity campaign was rounded 
out by radio appearances of the project director, and by direct 
mailings of test brochures and application forms to all state 
high school counselors, and to all persons applying^ at that 
time for admission to any of the California State Qolleges and 
Universities. In all, approximately 55,000 registration- 
information forms distributed.* 

Media coverage of the project following the May examination 
focused primarily on the t^st re;§ults. For example, the test 
outcomes r.eported by the Chancellor's Office in -a news release 
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dated June 28, 1973, v^re carried in the Los Anqele-s Times on 
■the following day under the byline of William Trombley who 
referred to the success of the 1973 test and the possibilities 
of refunding the program in 1974. The Los Angeles, Herald 
Examiner carried an editorial on. July 4, and many other 
newspapers throughout the state reported the test results. 

Public interest in the program seemed to remain fairly 
high. ' , 
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\ 73-10 FOR IMMEDIATE RELEASE February 25, 1973 

LOS ANGELES — The first systemwide program enabling students to earn 
academic credits without classroom attendance was announced today by 
Chancellor Glenn S. Dumke of The California State University and Colleges. 

The program will permit seniors in all California high schools to 
be tested May 12, 1973, oi) their knowledge of freshman English. Those 
who pass will receive credit equal to six semester units at any of the 
system's 19 campuses. - ^ 

Under the plan, incoming freshmen who demonstrate sufficient 
college-level ability will be able to move directly into advanced course 
work in English. 

Application forms will be mailed between March 1 and April 18 to 
students who have already -applied for fall 1973 admission. The forms 
must be returned by April 27, 1973. Forms also will be available from 
counselors in high schools. ^ 

The tests will be adfaFnistered on all California State University 
and Colleges campuses only the one day— Saturday , May 12. The campus on 
which the applicant takes the test need not be the one he plans to attend c 

"We are moving rapidly on the premise that many students come to our 
campuses with sufficient knowledge and ability to proceed directly into 
more advanced levels of acadetiic v/ork," Chancellor I^umke said. 

"The new testing program developed by representatives of our English 
faculties is an important example of our efforts to increase the options 
of higher education. It contains the hallmarks of much success." 

Dr. Edward M. White, chairman Of the Department ot English at 
California State College, San Bernardino, ' the project's director, said the 
test is divided into two 90-minute components with a break in between. 

The first component is an objective exam, the Colle'ge-Level EXam- - 

--MORE-- 
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ination Program (CLEP) Subject Examination in Analysis and Interpretation 
of Literature by the Educational Testing Service of Princeton, NtJr 

Sponsored by the College Entrance Examination Board/ obj ective-type 
CLEP tests were tried on a large-scale exploratory basis in 1971 with 
freshmen about to enter California State University, San Francisco, and 
California State College, Bakersfield. 

Although CLEP tests in the new project are not the same ''as thgse in 
the 1971 program, experience 'from the earlier project prompted wide*^ 
agreement among English f-aculties that fully adequate testing must contain, 
both objective and written portions. - * 

Thus/ the test's second component will^ be in essay form. * It will 
consist of two questions composed for students by English professors 
throughout The California State University and Colleges. 

Students taking the exam will pay a $15 fee, the standard cost for 
the CLEP test -alone. The system's Fund for Innovation and Improvement 
t> will finance the project's administration and the separate essay component. 

Results of the CLEP portion of the test will — at the option of the ^ 
student — be sent to other institutions as well as those-,in The California 
State University and Colleges. 

However, only the 19 campuses, in the State University and Colleges 
system will be provided, a list of students who pass bTrth parts of the 
exatn. These students, upon registration, can claim — and will automatically 
receive — academic credit. 

Passing scores will be based on average performance of "C" students 

who hav.e actually completed six semester units of freshman English. 

Students not attaining this level will not be penalized, however, as none 

of their ^names will be forwarded to admissions offices. • 

"We expect 3,000 to 4,000 students will take the examination," 
* « * 

Dr. White said. "Many able high school seniors shduld pass, if they have 

done enough reading and writing." 

Dr. White noted that, although English professors usualLy oppose mass 
testing programs, this one- has received unanimous approval of English 
Department chairmen because of its essay component and the high quality 
of the objective-type CLEP test. 

"The entire process is being supervised by English professors," he 
said. "The professors hc\ve ful?. ronfidence in this project and urge 
all 5tudentc who J:ecl th2y might pass to take advantage of this 
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innovative opportunity to gain a head start in college." 

In addition, tHe project has the endorsement of the Chancellor's 
Council of Presidents and the Academic Senate of The California State . 
University and Colleges. 

Dr. White, as project director, will be assisted by Dr. Richard Lid, 
chairman of the California State University, Northridge, English 
Department. Dr. William Abbott, chief test officjsr at California State 
University, Lohg Beach, will be coordinator of test a^dministration. 

Dr. White s^id students taking the examinaiiion will be informed of 
the results in two stages. They will receive results of the CLEP portion 
IN June from Princeton, N.J., and the combined results of both CLEP and 
essay components in July or August. 



NOTE: For additional information please contact Public Affairs, Office 
of the Chancellor (213) 938-2981, Ext\ 231, or Dr. Edward M. 
White, Cgc, San Bernardino (714) 8'S7-6311, Ext'^ 597. 
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73-46 FOR IMMEDIATE ^LEASE June 28 , 1973 

/ ^ 

LOS ANGELES — More thaft 1,300 newly graduated high school 
seniors will receive academic credit for freshman English upon entrance 
in The ^California State Ur^ersity and Colleges, Chancellor Glenn S. 
Dumke announced today. 

The seniors 'are among approximately 4,100 who last month took 
the, California State University English Equivalency Test, the first exam 
of its kind administered under auspices of the 19-campus system. 

By passing the two-part examination, students can receive credit 
equal to six semester units at any of the system's 14 universities and . 
five colleges 

/ • Thus, they ban move directly into' advanced coursework in E^iglish 



and — if they choosfe to do so — shorten the usual class attendance time 
required for graduation. 

The examination program yWas supported in part by the CSUC Fund 
for Innovation, created" by the Legislature last -yea^r in respo1?se to 
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Chancellor Dumke 's 1971 proposals for New Approaches to Higher Education. . 

English professors throughout the system devised and conducted 
the testing plan under the directiori of Dr.. Edward M.. White of California. 
State College, San Be'rnax^iino, and Dr. Richard Lid of California State 
University, Northridge. , ^ 

i r.-.MORE-- 
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"The results are most encouraging in that many students will, not 
be required to duplicate work they have had in high school and can use 
their time in higher education more effectively," Chancellor Dumke said. 

"I am pleased with both the results of the examijiation and the 
adherence to established academic principles which was strongly evidenced 
in this unique testing program." 

Dean David H. Provost of the Division of New Program Development 
and Evaluation , which monitors the system's, approximately 50 pilot projects 
in educational innovation, said students who took the test will be 
notified of their eligibility to receive academic credit in letters being 

r 

mailed this week. 

Congratulatory letters to the some 1,343 students eligible for 
academic credit- noted that they "have demonstrated the same level of 

ability many students achieve only after a full year's work in- college 

J 

English." 

The test, divided into two 90-minute components, was administered 
May 12 at all system campuses. 

The fitst component was an objective exam, the College-Level 

Examination Program (CI.EP) Subject Examination in Analysis dnd 

Interpretation of Literature. The CLEP portion was developed by the 

Educational Testing Service of Princeton, N.J.., for the College Entrance 

t 

Examination Board. 

Dr. White said California students performed very well on the CLEP 
component; ^€he -test^ group, ' without the codlege course/ approached^ the 
national norm based on students who have cor\pl^:dd ^fie college course. 

The test's second componeht required studej/ts to provide essay 

— MORE— 
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responses to two questions designed by English professors £rom The 
California State University and Colleges. 

Seventy-five English professors graded the essays during a ^ 
four-day session this month. Essays were read by four faculty 
members, working independently and without knowledge of grades assigned 
by the other readers. 

\ * 
Upon completion of the individual grading process, the ratings 

of all four readers formed the basis for assignment of a final gr'ade. 

Combination of scores on the CLEP component and the essay grades 
resulted in the determination of whether students qualified for academic 
credit in the State University and Colleges. 

Students taking the examination paid a $15 fee, the standard 
cost for the CLEP test alone. The system's Fundfor Innovation financed 
the project's administration and the essay component. 

# 

NOTE : For additional information please contact Public Affairs, Office 
of the Chancellor (213) 938-2981 , Ext. 231, or Dr. Edward M. White, 
CSC, San Beirnardino (714) 887-6311, .Ext. 597. 
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College-Level Equivalency Exams in English Draw Fire 



By Edward R. Weldlein 

The use of standardized tests to allow college 
freshmen to skip introductory courses is being 
Subjected lo^ ihcreasing scrutiny, particularly 
by professors' of English. 

The current debate over credit by examina- 
tion recalls faculty ooposttion in the iP^O's and 
early I960's to the advanced-placement tests of 
the College Ertrarfce Fxaminafion Board 

This lime, however, a newer ^e^cs of College 
Board tests— the College Level Examinaiion 
Program fci,rp>— is being looked at. 

The CLEP tests were instituted in 1 965 by the 
College Board as a way to provide academic 
ccfJification for knowledge that people picked 
up outsfdc of college classrooms 

That use of the test has been heavily pro- 
moted. In one clep television commercial, a 



tall, bearded job applicant named Lincoln 
turned away by a sleazy employment agent be- 
cause he does not have a college degree — de- 
spite Lincoln'* plea that he has read a lot, "son 
lOf on my own.** 

Aj/oiding Freshman Requirements 

idea of using ci i.p to give academic 
credit to contemporary Lincolns has been gen- 
erally applauded, but a growing ' number of 
colleges have also begun to use the tests to 
allow incoming freshmen to avoid normal 
freshman requfrements. such as English com- 
position. 

Some history professors, mathematics pro- 
fessors, and a few others have critici/ed this 
latter use of cua\ but English professors have 
been most vocal, and have organized several 



conferences ihis year to consider how to re- 
spond to the burgeoning practice of credit by 
examination. 

Their concern is hardly surprising. Already 
the profession is wary, because many colleges 
have been dropping or reducing freshman- 
English requirements, thereby at least poten- 
tially endangering the jobs of some English 
professors»Js'ow cli p has the further potential 
of cnablmg htige numbers of freshmen to skip 
many of the freshriian-English courses that still 
are required. 

These fears have combined with skepticism 
that an objective, multiple-choice test can really 
measure a student's writing ability, and with 
distrust of administrators who impose credit- 
by-examination programs. The concern sur- 
Continued on Page 6, Column 3 
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Continued from Page 1 
faced m Cattfornia about a year ago. 
when ihc California Stale Unsvcrsity 
and Colleges, in an experimenl, gave 
a battery of five clep tests to entering 
students at two of its 19 campuses. 

Of the 873 students who took the 
tests aj California State University at 
San Francisco, 33 1 — 38 per cent — 
passed all five and became "instant 
sophomores." About 10 per cent of the 
148 students who took the ;est at Cali< 
forma State College at Bakersfield also 
scored well enough to skip the fresh- 
man year. 

Several of the students at Bakers* 
field were quickly counseled by the 
English department, which had given^ 
its own written placement test, to take 
remedial English before they could 
expect to take even the regular fresh- 
man course. 

Experiences like that, and general 
faculty objections that the English 
section of the tests did not do well in 
measuring students* writing abilities, 
led to the appointment of a university 
committee to study clep. Last fall» 
the committee produced a 25-page re- 
port, and this Spring it will conduct 
its own experiment with a clhp test. 

Few professors challenge the basic 
idea of a test like clcp. 

*'No one should be asked to repeat 
work in college that he has mastered; 
be should receive credit for what he 
knows, and proceed to appropriate 
levels of learning.'* says Ed\vard M. 
White, chairman of the English depart- 
ment at, California State College at 
San Bernardino and author of the 
report on clep. 

'A DlMSter" 

But some professors do insist that 
some CLEP tests are inadequate. "The 
use of the present ci.EP objective tests 
as a means of exempting a whole year 
of freshman English is a disaster," 
charges James A. Parrtsh, English de< 
partment chairman at the Unjversity 
of South Florida. 

The CLEP tests have been used 
on a mass basis elsewhere without 
creating as much furor as they did 
in California. 

At the University 6f Utah, about 
1^200 students were able to trim a full 
year off the time it will take them to 
earn a degree. Although Utah has 
been probably the largest single user of 
CLEP, faculty criticism of the 'tests 
there ha5 been restrained. 

Some of the criticism nationwide 



reflects little more than a common 
professorial aversion to standardized 
tests. Mr. White's reppn tries to com^ 
bat such perceptions. He points to re- 
search that shows that' "one good o6- 
jective test will correlate more highly 
with a student's writing ability « . . 
than will one good essay test." 

That is more or less accepted 
gospel among psychometricians. the 
people who put objective tests to- 
gether Less widely held is a view Mr. 
White puts forth as almost a man- 
datory corollary: "The combination 
of objective items (which measure 
accurately some skills involved in 




Edward M. White: 
"No one should be asked 
to repeat work in college 
that he has mastered.'' 



writing) with an essay (which meas- 
ures directly, if somewhat less ac- 
curately, the writing itselO proved to 
be- more .^valid than either typjc of 
item alone." 

Ilic statement comes, in fact, from 
The Measurement of Writm}^ Ability, 
published in 1966 by the Educational 
Testing Service, the- semi-autonomous 
agency that the College Board con- 
tracts with to write, administer, and 
score CLEP and ether tests. 

Some h.T.s. ofTicials question, how- 
ever, whether the benefits of including 
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essays with the objective te.sts are 
worth the costs of grading the*essays. 
which Mr. White admits is a *'hide- 
ously expensive" undertaking. 

A puestion of Costs 

He and his colleagues hope to show 
that the essays are worth the $6 or 
more it costs to grade each one. 

It IS a cost the College Board itself 
cannot hope to bear The clep pro- 
gram has already consumed S3.1- 
million in grants from the Carnegie 
Corporation and at least an equal 
amount in board funds. The number 
of people , taking the tests is mush- 
rooming, however. There were about 
20,000 candidates last year and this 
year the board expects 55.000. 

This month, more than 30.000 ap- 
plication forms for a special *'Cali- 
fornia State University English 
Equivalency Test" were mailed to / 
California high schools. The test, to/ 
be given in May. is essentially an ob« 
jective CL'EP test on the "analysis arid 
interpretation, of Jiterature.*' together 
with an essay question put together 
by the state system's English pro- 
fessors. 

Mr. White estimates t]jat 4,000 
high school students^wi)/$ign up. to 
take the* test.. Those wffa get a com- 
bined grade of C or better will re- 
ceive six semester-units of credits in 
English at the college and university 
system's- 1 9 campuses. 

More Rigorous Test 

Students will pay the normal Sl5. 
CLEP fee to take the test. Those fees 
will go to the College Board. The 
state system ha<i also allocated' 
$40,000 to" cover the grading costs of 
$6 per test plus other expenses. 

Implicit, in this plan developed by 
Mr White and his fellow English 
professors is the belief that the new. 
more rigorous clep test will not per- 
mit as many as 38 per cent of the 
students who take it .\.6 become 
"instant sophomores." *. 

"If tiquivalcncy becomes one more 
mechanical device to turn education 
into processing, we will have done 
our students and our society a sig- 
nificant disservice, even if we have 
saved them some cash," Mr. White 
says. 

Although CLEP officials express 
much greater confidence in their 
whole range of tests than does Mr. 
White, they have also taken some of 
his comments 10 heart. Albert Serling. 
CLEP's program director at Educa- 
tional Testing Service, calls Mr, 
White's report ''about the best thing 
,that has ever been done in the field 
of English testing." 

But CLEP officials may have some 
difficulty in convincing professors that 
their jobs are not jeopardized by the^ 
tests. For instance. James Ballowe. 
chairman of the Rnglish department 
at Bradley University, states** 'The 
examinations inevitably- lead to the* 
^ cubing down o( staff in large fresh- 



